-- Logs begin at Tue 2022-08-02 09:26:16 UTC, end at Tue 2022-08-02 11:10:14 UTC. -- Aug 02 09:59:51.095819 dv-cinder-847536-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 02 09:59:51.104495 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 2 09:59:51 2022] *** Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: nodename: dv-cinder-847536-4-iscsi Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: machine: x86_64 Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: clock source: unix Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: pcre jit disabled Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: detected number of CPU cores: 4 Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: current working directory: / Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: detected binary path: /usr/bin/uwsgi-core Aug 02 09:59:51.107092 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: your processes number limit is 47680 Aug 02 09:59:51.107853 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: your memory page size is 4096 bytes Aug 02 09:59:51.107853 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: detected max file descriptor number: 2048 Aug 02 09:59:51.107853 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: lock engine: pthread robust mutexes Aug 02 09:59:51.107853 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: thunder lock: enabled Aug 02 09:59:51.107853 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 02 09:59:51.107853 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 02 09:59:51.148082 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: Python main interpreter initialized at 0x5643ebb979c0 Aug 02 09:59:51.148082 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: python threads support enabled Aug 02 09:59:51.148082 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: your server socket listen backlog is limited to 100 connections Aug 02 09:59:51.148082 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: your mercy for graceful operations on workers is 90 seconds Aug 02 09:59:51.148437 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: mapped 403077 bytes (393 KB) for 2 cores Aug 02 09:59:51.148510 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: *** Operational MODE: preforking *** Aug 02 09:59:51.148791 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: *** uWSGI is running in multiple interpreter mode *** Aug 02 09:59:51.148791 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: spawned uWSGI master process (pid: 80460) Aug 02 09:59:51.148997 dv-cinder-847536-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 02 09:59:51.149682 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: spawned uWSGI worker 1 (pid: 80466, cores: 1) Aug 02 09:59:51.149899 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: spawned uWSGI worker 2 (pid: 80467, cores: 1) Aug 02 09:59:51.152780 dv-cinder-847536-4-iscsi devstack@keystone.service[80460]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 02 09:59:56.704832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:59:56.704832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: from cryptography.utils import int_from_bytes Aug 02 09:59:56.705625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:59:56.705625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: from cryptography.utils import int_from_bytes Aug 02 09:59:56.705945 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:59:56.705945 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: from cryptography.utils import int_from_bytes Aug 02 09:59:56.706406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:59:56.706406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: from cryptography.utils import int_from_bytes Aug 02 09:59:56.756748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 09:59:56.756952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 09:59:56.757039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 09:59:56.757128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 09:59:56.757210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] command line args: [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 09:59:56.757286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] command line args: [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 09:59:56.757393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 09:59:56.757510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 09:59:56.757593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ================================================================================ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 09:59:56.757704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ================================================================================ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 09:59:56.757839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] admin_token = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.757958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] admin_token = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config_dir = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config_dir = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config_source = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] debug = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config_source = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] debug = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.758847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.759972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760274 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_config_append = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_config_append = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_dir = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_dir = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.760979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.761932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.762951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.763949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.764060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.764217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.764324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.764726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.764838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.764920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] publish_errors = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] publish_errors = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.765862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] standard_threads = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] standard_threads = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.766910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_journal = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_journal = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_json = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_json = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_stderr = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_stderr = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_syslog = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_syslog = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.767893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:59:56.768065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.768146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.768306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.768382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.768562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.768638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.768800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.768884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.769855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.770984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.771060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.771238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.771318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.771488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.771589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.771737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.771824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.772498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.772585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.772760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.772850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.773825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.774977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.775971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.776922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.777983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.778992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.779937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.780968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.781913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.782873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.783869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.784993 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.785959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.786989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.787148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.787267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.787417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.787509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.787705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.788358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.788469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.788552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.788706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.788795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.788942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.789995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.790979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.791904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.792846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.793897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.external = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.external = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.794874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.password = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.password = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.token = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.token = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.795967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.796940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.797986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.798946 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.799984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.800186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.800263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.800433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.800511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.800698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.800785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.800942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.801927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.802969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.803143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.803326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.803430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.803509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.803690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.803770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.803932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.804919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.805963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.806930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.807911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808585 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.808935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809796 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.809874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810823 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.810987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.811973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.812936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.813957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.814929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.815928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816149 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.816850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.817995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.818967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.819941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.820017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.820189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.820275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.820433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.820508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.820800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.820877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.821861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.822982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.823980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.824943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.825971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.826960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.827953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.828909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.829903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830563 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.830888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.831885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.832054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.832134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.832479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.832554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.832719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.832802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.832972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.833959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.834035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.834213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.834299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.835978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.836972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.837989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.838984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.839979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.840925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.841936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.842926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.843970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.844990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845235 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.845944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.846028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.846181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.846258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.846414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.846492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:59:56.846586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 09:59:56.846671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 09:59:56.876497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.876792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.876886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.876965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.877060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.877196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.885924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.886171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.915772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.915990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.916096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.916228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.968860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:56.969232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.019465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.019700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.019925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.020168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086540 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.086999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.087128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.087227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.087378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.087600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.110123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.110360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.110584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.110668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.110808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.110888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.111029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.111112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.111277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.111359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.111499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.111591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.111718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.112193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.123301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.123678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.136599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.136816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.136902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.137927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.138007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.138144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:59:57.155331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.155437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.155544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:59:57.155640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:59:57.155882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.155963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.156086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:59:57.156166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:59:57.156354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.156435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.156611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.156690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.156866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.156944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.157120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.157199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.167370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.167465 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.167648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.167742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.167924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.168044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.168196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.168280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.175069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.175171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.179887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.179984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.180172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.180255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.180454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.180538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.180746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.180830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.181882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.182066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.182147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.182335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.182414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.182595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.182676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.205924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.206028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.207427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.207527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.207710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.207794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.212871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.212967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.213148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.213232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.213429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.213512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.220086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.220177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.220369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.220452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.224471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.224562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.224774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.224892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.225999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.226184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.226265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.226457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.226541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.242553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.242651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.242830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.242927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.246743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.246830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.249931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.250015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.250230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.250314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.254625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.254753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.258398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.258503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.261553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.261646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.261849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.261933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.262144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.262230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.262427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.262510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.262709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.262808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.262998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.263091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.278895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.279013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.279179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.279288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.279467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.279550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.279728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.279824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.286549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.286635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.288677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.288776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.288949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.289034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.291782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.291875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.292064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.292147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.292345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.292430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.292627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.292718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.292916 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.293009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.304293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.304384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.304574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.304658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.304852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.304935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.305126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.305209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.305402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.305494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.305679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.305762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.305952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.306035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.321892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.322058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.324921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.325083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.327877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.328058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.330435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.330629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.332030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.332242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.332327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.332531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.332615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.332814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.339110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.339315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.340688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.340904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.343617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.343828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.343911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.344123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.344223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.344409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.344499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.344695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.352956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.353246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.353330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.353528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.353611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.353807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.360346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.360625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.360708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:59:57.360907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.360989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.361187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.361272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.361466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.361548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.361743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.362099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.362276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.362367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.362545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.362627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.362811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.362907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.363206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.363291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.363490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.363572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.363754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.363838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.364025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.364107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.364282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.364372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.364541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:59:57.398175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5643ebb979c0 pid: 80467 (default app) Aug 02 09:59:57.398376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5643ebb979c0 pid: 80466 (default app) Aug 02 09:59:57.401658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e0f45-a093-4109-8904-07425d76426b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:57.401949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e0f45-a093-4109-8904-07425d76426b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:57.402227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e0f45-a093-4109-8904-07425d76426b None None] PATH_INFO: `/v3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:57.406578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Tue Aug 2 09:59:57 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:06.248505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13030e-0ca8-49d8-b160-51a0040a15ac None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.249050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13030e-0ca8-49d8-b160-51a0040a15ac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.249050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13030e-0ca8-49d8-b160-51a0040a15ac None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.253571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:06.258995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.259485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.259647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.331296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: INFO dbcounter [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] Registered counter for database keystone Aug 02 10:00:06.351196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG oslo_db.sqlalchemy.engines [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80467) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 10:00:06.357463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writer thread running {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 10:00:06.539352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG passlib.handlers.bcrypt [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80467) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 10:00:06.539786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG passlib.handlers.bcrypt [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80467) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 10:00:06.595274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:06.598343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:06.729822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 10:00:06.884292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13030e-0ca8-49d8-b160-51a0040a15ac None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:06.955442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: INFO dbcounter [None req-fa13030e-0ca8-49d8-b160-51a0040a15ac None None] Registered counter for database keystone Aug 02 10:00:06.977411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG oslo_db.sqlalchemy.engines [None req-fa13030e-0ca8-49d8-b160-51a0040a15ac None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80466) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 10:00:06.983695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writer thread running {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 10:00:07.103809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13030e-0ca8-49d8-b160-51a0040a15ac None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:07.108130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ecb30-bf11-4833-8102-5564f89a72f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:07.108769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ecb30-bf11-4833-8102-5564f89a72f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:07.109117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ecb30-bf11-4833-8102-5564f89a72f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:07.109372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ecb30-bf11-4833-8102-5564f89a72f2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:07.118621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ecb30-bf11-4833-8102-5564f89a72f2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:07.169798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d07ecb30-bf11-4833-8102-5564f89a72f2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 10:00:07.170076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ecb30-bf11-4833-8102-5564f89a72f2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:07.181124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Tue Aug 2 10:00:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:00:07.187037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:07.201385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dc6e2-d0a8-49ca-a5f5-7c8c0fb0fa9d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:07.205763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599895d3-0101-4fa3-b7ba-5134f85afe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:07.206900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599895d3-0101-4fa3-b7ba-5134f85afe94 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:07.207181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599895d3-0101-4fa3-b7ba-5134f85afe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:07.207478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599895d3-0101-4fa3-b7ba-5134f85afe94 admin admin] PATH_INFO: `/v3/projects/737b03aeb0594b48a03adad9f790a347` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:07.208975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599895d3-0101-4fa3-b7ba-5134f85afe94 admin admin] RBAC: Authorizing `identity:get_project(project_id=737b03aeb0594b48a03adad9f790a347)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:07.259858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING keystone.common.rbac_enforcer.enforcer [None req-599895d3-0101-4fa3-b7ba-5134f85afe94 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 10:00:07.260466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599895d3-0101-4fa3-b7ba-5134f85afe94 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:07.265133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Tue Aug 2 10:00:07 2022] GET /identity/v3/projects/737b03aeb0594b48a03adad9f790a347 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:09.270454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c985-20ef-45f1-9fbc-d465a81eb54a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:09.271032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c985-20ef-45f1-9fbc-d465a81eb54a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:09.271032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c985-20ef-45f1-9fbc-d465a81eb54a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:09.275125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:09.280542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac61a-62ac-412a-83ab-ebd60bcecdbd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:09.280992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac61a-62ac-412a-83ab-ebd60bcecdbd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:09.281086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ac61a-62ac-412a-83ab-ebd60bcecdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:09.301678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9d1ac61a-62ac-412a-83ab-ebd60bcecdbd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:09.303832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9d1ac61a-62ac-412a-83ab-ebd60bcecdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:09.311793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 10:00:09.333685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c985-20ef-45f1-9fbc-d465a81eb54a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:09.345597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c985-20ef-45f1-9fbc-d465a81eb54a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:09.349743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68a132-4185-412e-850e-c7cd29f02de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:09.350860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68a132-4185-412e-850e-c7cd29f02de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:09.351142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68a132-4185-412e-850e-c7cd29f02de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:09.351447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68a132-4185-412e-850e-c7cd29f02de5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:09.359110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68a132-4185-412e-850e-c7cd29f02de5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:09.360923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68a132-4185-412e-850e-c7cd29f02de5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:09.370597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Tue Aug 2 10:00:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:00:09.376281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ac61a-62ac-412a-83ab-ebd60bcecdbd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:09.385913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ac61a-62ac-412a-83ab-ebd60bcecdbd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:09.390363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ebad5-2012-4f4f-91f4-60fa0b4ed357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:09.391015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ebad5-2012-4f4f-91f4-60fa0b4ed357 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:09.391337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ebad5-2012-4f4f-91f4-60fa0b4ed357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:09.391625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ebad5-2012-4f4f-91f4-60fa0b4ed357 admin admin] PATH_INFO: `/v3/users/f7156356453b494e8893e4adc4a9ad81` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:09.393014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ebad5-2012-4f4f-91f4-60fa0b4ed357 admin admin] RBAC: Authorizing `identity:get_user(user_id=f7156356453b494e8893e4adc4a9ad81)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:09.394866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ebad5-2012-4f4f-91f4-60fa0b4ed357 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:09.396578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Tue Aug 2 10:00:09 2022] GET /identity/v3/users/f7156356453b494e8893e4adc4a9ad81 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:11.470961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed83da0-3b1d-4605-8f25-a459f1db0a4c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:11.471533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed83da0-3b1d-4605-8f25-a459f1db0a4c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:11.471533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed83da0-3b1d-4605-8f25-a459f1db0a4c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:11.472903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:11.478704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7adff7-9423-435b-b4ee-86593294d941 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:11.479173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7adff7-9423-435b-b4ee-86593294d941 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:11.479296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7adff7-9423-435b-b4ee-86593294d941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:11.499460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ce7adff7-9423-435b-b4ee-86593294d941 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:11.500728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ce7adff7-9423-435b-b4ee-86593294d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:11.501750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de4e6-a185-485b-ac1b-5e4b5682e40a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:11.502048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de4e6-a185-485b-ac1b-5e4b5682e40a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:11.502331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de4e6-a185-485b-ac1b-5e4b5682e40a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:11.503714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:11.509160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:11.509454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:11.509739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:11.558253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG passlib.handlers.bcrypt [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80466) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 10:00:11.558848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG passlib.handlers.bcrypt [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80466) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 10:00:11.586478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:11.605392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:11.607035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:11.609122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7adff7-9423-435b-b4ee-86593294d941 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:11.633377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:11.708708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:11.712624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7adff7-9423-435b-b4ee-86593294d941 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:11.731502 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:11.748735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:11.749403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:11.749694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:11.749979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:11.761608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:11.770658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:11.772340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:11.773172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:11.778200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:11.835750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e420a-6014-4bca-b7f1-3709e5953047 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:11.838074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:11.862808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:11.875374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:11.875374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:11.875374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:11.875763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:11.880527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e76950-355f-49e0-be01-59a2a6e239a0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:11.915106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:11.915810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:11.916111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:11.916392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:11.917136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:11.917449 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:11.918877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:11.931755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:11.937329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:11.944100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:11.945650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:12.044261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:00:12.046072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a7c6f-bc58-4aa2-aa62-9ba502babde3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:12.049921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:12.052880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:12.157663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e208-c339-4c03-a950-c7c614f9cbeb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:12.188505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:12.189167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:12.189453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:12.189732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] PATH_INFO: `/v3/users/f7156356453b494e8893e4adc4a9ad81` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:12.190932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] RBAC: Authorizing `identity:get_user(user_id=f7156356453b494e8893e4adc4a9ad81)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:12.192451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:12.194618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:11 2022] GET /identity/v3/users/f7156356453b494e8893e4adc4a9ad81 => generated 247 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:12.195590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:12.196470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:12.196758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:12.197035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:12.200220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:12.215782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:12.216628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:12.218125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:00:12.236617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:12.237873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:12.238794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:00:12.238794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:00:12.238794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:12.238794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:12.238794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:12.246288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:12.309535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:12.314069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27c0a-9c4a-45f1-b97f-38ef5203ed14 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:12.348958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:12.349606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:12.350047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:12.350334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:12.351533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:12.353011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:12.353853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2988a3-2bb5-4928-901a-4cc97aaf8609 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:12.354607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:12.360506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:12.388327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec7d11-7685-4032-b534-66c648c43ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:12.388962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7d11-7685-4032-b534-66c648c43ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:12.389250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7d11-7685-4032-b534-66c648c43ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:12.389531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7d11-7685-4032-b534-66c648c43ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:12.390446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7d11-7685-4032-b534-66c648c43ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:12.390762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7d11-7685-4032-b534-66c648c43ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:12.392500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7d11-7685-4032-b534-66c648c43ba2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:12.410944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:12.446077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:12.467980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14321c7-3b04-46d7-a7b5-a81c2884699b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:12.506059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f086a0-9c5c-409a-abc8-a4b201b94e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:12.507420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f086a0-9c5c-409a-abc8-a4b201b94e63 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:12.507716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f086a0-9c5c-409a-abc8-a4b201b94e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:12.508004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f086a0-9c5c-409a-abc8-a4b201b94e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:12.508834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f086a0-9c5c-409a-abc8-a4b201b94e63 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.domain.id=default, user.id=f7156356453b494e8893e4adc4a9ad81) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:12.509150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f086a0-9c5c-409a-abc8-a4b201b94e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:12.510652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f086a0-9c5c-409a-abc8-a4b201b94e63 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:12.518471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Tue Aug 2 10:00:12 2022] GET /identity/v3/role_assignments?user.id=f7156356453b494e8893e4adc4a9ad81&scope.domain.id=default&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:14.923662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642885e7-5a6e-4dbe-87ce-c8938d8ea67c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:14.924145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642885e7-5a6e-4dbe-87ce-c8938d8ea67c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:14.924145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642885e7-5a6e-4dbe-87ce-c8938d8ea67c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:14.925490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:14.927173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb4fe-1d2c-483b-8678-2c462abf838b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:14.928563 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb4fe-1d2c-483b-8678-2c462abf838b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:14.929065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb4fe-1d2c-483b-8678-2c462abf838b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:14.930628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 12/23] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:14.932088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00df7d-0b99-4275-99e8-b6f762e6c2e3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:14.932596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00df7d-0b99-4275-99e8-b6f762e6c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:14.933072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00df7d-0b99-4275-99e8-b6f762e6c2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:14.937666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad67ef-10c1-4868-b287-e6fd69d00abe None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:14.938207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad67ef-10c1-4868-b287-e6fd69d00abe None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:14.938684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad67ef-10c1-4868-b287-e6fd69d00abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:14.981806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6cad67ef-10c1-4868-b287-e6fd69d00abe None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:14.983228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6cad67ef-10c1-4868-b287-e6fd69d00abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:14.992305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-da00df7d-0b99-4275-99e8-b6f762e6c2e3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:14.993874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-da00df7d-0b99-4275-99e8-b6f762e6c2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:15.111618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 13/24] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:15.115321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8bebd-8adf-4862-80f4-1747462d79c9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.116167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8bebd-8adf-4862-80f4-1747462d79c9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.116456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8bebd-8adf-4862-80f4-1747462d79c9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.122593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 14/25] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:15.128478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d5715-cf6c-484c-8495-ff0d158c9a50 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.128772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d5715-cf6c-484c-8495-ff0d158c9a50 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.129058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d5715-cf6c-484c-8495-ff0d158c9a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.165398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 12/26] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:15.166284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00df7d-0b99-4275-99e8-b6f762e6c2e3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:15.167013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-586d5715-cf6c-484c-8495-ff0d158c9a50 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:15.172439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-586d5715-cf6c-484c-8495-ff0d158c9a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:15.235416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:15.291827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 15/27] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:15.296167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d5715-cf6c-484c-8495-ff0d158c9a50 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:15.304217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00df7d-0b99-4275-99e8-b6f762e6c2e3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:15.305773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:15.351380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114e214-4f23-448d-a5d1-5703eab923fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:15.352432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114e214-4f23-448d-a5d1-5703eab923fb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.352947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114e214-4f23-448d-a5d1-5703eab923fb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.353423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114e214-4f23-448d-a5d1-5703eab923fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.366518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114e214-4f23-448d-a5d1-5703eab923fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a114e214-4f23-448d-a5d1-5703eab923fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:15.373553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:15.374400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:15.375136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:15.377524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 13/28] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:15.379091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ad94-ce92-41e9-9bbf-56242daec8b9 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.379611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ad94-ce92-41e9-9bbf-56242daec8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.380097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ad94-ce92-41e9-9bbf-56242daec8b9 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.386101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 14/29] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:15.387201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ad94-ce92-41e9-9bbf-56242daec8b9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:15.432708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d5715-cf6c-484c-8495-ff0d158c9a50 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:15.472029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:15.472547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.472842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.473126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.473852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:15.475909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:15.501264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ad94-ce92-41e9-9bbf-56242daec8b9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:15.623015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:15.652271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd2da-cb04-4906-baa8-cb7a668a5203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:15.653017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd2da-cb04-4906-baa8-cb7a668a5203 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.653300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd2da-cb04-4906-baa8-cb7a668a5203 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.653579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd2da-cb04-4906-baa8-cb7a668a5203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.665179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dd2da-cb04-4906-baa8-cb7a668a5203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:15.666730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dd2da-cb04-4906-baa8-cb7a668a5203 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:15.678985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 16/30] 10.222.0.49 () {60 vars in 1131 bytes} [Tue Aug 2 10:00:15 2022] POST /identity/v3/roles => generated 219 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:15.680348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-958dd2da-cb04-4906-baa8-cb7a668a5203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:00:15.682698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:15.683552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:15.684785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:00:15.684785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:00:15.684785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:00:15.684785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:15.684785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 15/31] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:15.688488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc64db-dde9-4368-b98a-9082a08765ec None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.689098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc64db-dde9-4368-b98a-9082a08765ec None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.689674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc64db-dde9-4368-b98a-9082a08765ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.716603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b9dc64db-dde9-4368-b98a-9082a08765ec None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:15.717838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9dc64db-dde9-4368-b98a-9082a08765ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:15.765192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:15.835617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1c4ef-132d-4720-84a1-b3cf01bbfa50 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:15.881386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 16/32] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:15.884044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc64db-dde9-4368-b98a-9082a08765ec None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:15.884886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:15.885811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:15.886800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:15.889023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:15.889972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:15.890493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:15.899607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:15.911756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:15.913676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:16.014263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:16.030215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc64db-dde9-4368-b98a-9082a08765ec None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.057961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1077045-951f-465e-ae16-89cfeea815a8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.069714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.070459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.070753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.071036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.071782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:16.072446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.073628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.088000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 17/34] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:16.088942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:16.094969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:16.107109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.109332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.109332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.109332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.109741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.111714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.179437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 18/35] 10.222.0.49 () {60 vars in 1131 bytes} [Tue Aug 2 10:00:15 2022] POST /identity/v3/roles => generated 225 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:16.179437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:16.224230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692be50-5ee6-431c-b191-b2d4fbc80e10 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.267498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25061644-43ca-46cb-81cb-cbaed5e6e2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.268335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25061644-43ca-46cb-81cb-cbaed5e6e2ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.268689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25061644-43ca-46cb-81cb-cbaed5e6e2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.269034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25061644-43ca-46cb-81cb-cbaed5e6e2ac None admin] PATH_INFO: `/v3/users/f7156356453b494e8893e4adc4a9ad81` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.270980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25061644-43ca-46cb-81cb-cbaed5e6e2ac None admin] RBAC: Authorizing `identity:get_user(user_id=f7156356453b494e8893e4adc4a9ad81)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.273250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25061644-43ca-46cb-81cb-cbaed5e6e2ac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.275449 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 18/36] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:16 2022] GET /identity/v3/users/f7156356453b494e8893e4adc4a9ad81 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:16.276895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb1024-db17-4674-899c-629c5cd488a7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.277278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb1024-db17-4674-899c-629c5cd488a7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.277627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb1024-db17-4674-899c-629c5cd488a7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.279509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 19/37] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:16.285247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb1024-db17-4674-899c-629c5cd488a7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:16.311382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:16.340399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a37ec3-9531-4fc4-babd-1299d7c8e140 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.382653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9338613-56a1-4fb6-9128-d3809b4d5ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.383345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9338613-56a1-4fb6-9128-d3809b4d5ab6 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.383633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9338613-56a1-4fb6-9128-d3809b4d5ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.383914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9338613-56a1-4fb6-9128-d3809b4d5ab6 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.384622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9338613-56a1-4fb6-9128-d3809b4d5ab6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.386166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9338613-56a1-4fb6-9128-d3809b4d5ab6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.434789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb1024-db17-4674-899c-629c5cd488a7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.472479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7082022-9c4f-4fec-8fcd-6a267a333c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.473140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7082022-9c4f-4fec-8fcd-6a267a333c49 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.473433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7082022-9c4f-4fec-8fcd-6a267a333c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.473714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7082022-9c4f-4fec-8fcd-6a267a333c49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.475558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7082022-9c4f-4fec-8fcd-6a267a333c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.477039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7082022-9c4f-4fec-8fcd-6a267a333c49 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.478718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 20/38] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:16.480000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6378-bbe3-461b-bcc4-18954af2de3c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.480298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6378-bbe3-461b-bcc4-18954af2de3c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.480639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6378-bbe3-461b-bcc4-18954af2de3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.490056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 19/39] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:00:16 2022] POST /identity/v3/projects => generated 301 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:16.492326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b74a-288c-4042-824b-2543e3a5c2a6 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.492633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b74a-288c-4042-824b-2543e3a5c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.492921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b74a-288c-4042-824b-2543e3a5c2a6 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.494455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 20/40] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:16.499700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b74a-288c-4042-824b-2543e3a5c2a6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:16.501220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7d2e6378-bbe3-461b-bcc4-18954af2de3c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:16.502600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7d2e6378-bbe3-461b-bcc4-18954af2de3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:16.551902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:16.589959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 21/41] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:16.591312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f423-5ada-4d7d-b9e0-73e41dd3a755 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.591614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f423-5ada-4d7d-b9e0-73e41dd3a755 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.591900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f423-5ada-4d7d-b9e0-73e41dd3a755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.612891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fe02f423-5ada-4d7d-b9e0-73e41dd3a755 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:16.614655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fe02f423-5ada-4d7d-b9e0-73e41dd3a755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:16.626255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:16.635842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b74a-288c-4042-824b-2543e3a5c2a6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.682811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.683568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.684324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.684324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] PATH_INFO: `/v3/domains/default/users/f7156356453b494e8893e4adc4a9ad81/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.686660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f7156356453b494e8893e4adc4a9ad81, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.688179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.698593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 22/42] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:16.699686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02f423-5ada-4d7d-b9e0-73e41dd3a755 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:16.800021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02f423-5ada-4d7d-b9e0-73e41dd3a755 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.801440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 21/43] 10.222.0.49 () {58 vars in 1361 bytes} [Tue Aug 2 10:00:16 2022] PUT /identity/v3/domains/default/users/f7156356453b494e8893e4adc4a9ad81/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:16.802348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:16.842192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5916b-1442-4c18-bc9e-1c8eef77267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.842844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5916b-1442-4c18-bc9e-1c8eef77267e None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.843141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5916b-1442-4c18-bc9e-1c8eef77267e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.843454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5916b-1442-4c18-bc9e-1c8eef77267e None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.844204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5916b-1442-4c18-bc9e-1c8eef77267e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.845630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5916b-1442-4c18-bc9e-1c8eef77267e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.916492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:16.920756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5201c-26f8-4188-bcbd-a4c02c400aa0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:16.963543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:16.964316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:16.964683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:16.965041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:16.966319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:16.968023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:16.969749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 22/44] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:16.975589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:17.061735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 23/45] 10.222.0.49 () {60 vars in 1131 bytes} [Tue Aug 2 10:00:16 2022] POST /identity/v3/roles => generated 223 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:17.084053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4cff7-f0c9-4840-a693-f26545efe4b3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:17.124099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cd68c-f485-44e5-bd21-25b0c6328383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:17.124751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd68c-f485-44e5-bd21-25b0c6328383 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:17.125041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd68c-f485-44e5-bd21-25b0c6328383 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:17.125327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd68c-f485-44e5-bd21-25b0c6328383 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:17.126046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cd68c-f485-44e5-bd21-25b0c6328383 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:17.127578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cd68c-f485-44e5-bd21-25b0c6328383 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:17.215224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 23/46] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:00:16 2022] POST /identity/v3/projects => generated 312 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:18.830988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c0686-e19a-4aaa-994f-cfc2ec38da3a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.831510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c0686-e19a-4aaa-994f-cfc2ec38da3a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.831510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c0686-e19a-4aaa-994f-cfc2ec38da3a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.832863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 24/47] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:18.838594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bb822-7dc1-4a4c-93ba-0d4b62861dab None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.839037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bb822-7dc1-4a4c-93ba-0d4b62861dab None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.839165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bb822-7dc1-4a4c-93ba-0d4b62861dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.858069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2e7bb822-7dc1-4a4c-93ba-0d4b62861dab None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:18.859621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2e7bb822-7dc1-4a4c-93ba-0d4b62861dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:18.874789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:18.930256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 24/48] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:18.952315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c0686-e19a-4aaa-994f-cfc2ec38da3a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:18.956470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.062799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c0686-e19a-4aaa-994f-cfc2ec38da3a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.098438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951876-1928-4792-9066-5d23d725ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.099016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951876-1928-4792-9066-5d23d725ee20 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.099331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951876-1928-4792-9066-5d23d725ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.099614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951876-1928-4792-9066-5d23d725ee20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.110526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1951876-1928-4792-9066-5d23d725ee20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f1951876-1928-4792-9066-5d23d725ee20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:19.115792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:19.117093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:19.117846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 25/49] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:19.123300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bb822-7dc1-4a4c-93ba-0d4b62861dab None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.221739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bb822-7dc1-4a4c-93ba-0d4b62861dab None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.249365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085f4f-7a4b-49c9-bca7-570c4102d5ea None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.249654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085f4f-7a4b-49c9-bca7-570c4102d5ea None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.249934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71085f4f-7a4b-49c9-bca7-570c4102d5ea None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.251303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 26/50] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:19.256937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddb2f3-ff73-4bf2-a634-b3e06bb8ca2f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.257223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddb2f3-ff73-4bf2-a634-b3e06bb8ca2f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.257501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddb2f3-ff73-4bf2-a634-b3e06bb8ca2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.257696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.258341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.258634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.258921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.259694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:19.260013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.261419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.270186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 25/51] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:19.275918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c0ddb2f3-ff73-4bf2-a634-b3e06bb8ca2f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:19.276393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.277437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c0ddb2f3-ff73-4bf2-a634-b3e06bb8ca2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:19.289420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.337125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.345225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 27/52] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:19.368596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddb2f3-ff73-4bf2-a634-b3e06bb8ca2f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.378036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c2297-a412-41ff-8c35-ef97385c61a7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.413374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.414033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.414323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.414607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] PATH_INFO: `/v3/users/f7156356453b494e8893e4adc4a9ad81` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.415856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] RBAC: Authorizing `identity:get_user(user_id=f7156356453b494e8893e4adc4a9ad81)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.417342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.418980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 26/53] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:19 2022] GET /identity/v3/users/f7156356453b494e8893e4adc4a9ad81 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:19.424458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.468150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddb2f3-ff73-4bf2-a634-b3e06bb8ca2f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.503154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.503845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.504133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.504418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.505150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.505603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.507828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.509637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 28/54] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:19.515127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.528690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c2dc6-2e1f-46b6-a007-d9c169b86772 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.563975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.564634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.564921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.565212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.566393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.567929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.569856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 27/55] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:19.575602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.583242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.621699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d018df-b0bc-47ef-b1ee-73763f9a2ba1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.657442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018b78c-dcbc-4463-9431-062295587eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.658084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018b78c-dcbc-4463-9431-062295587eaa None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.658372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018b78c-dcbc-4463-9431-062295587eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.658657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018b78c-dcbc-4463-9431-062295587eaa None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.659390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018b78c-dcbc-4463-9431-062295587eaa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.660860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018b78c-dcbc-4463-9431-062295587eaa None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.678639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc15e2-559d-44b7-94d8-c301634b23bc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.713359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd2c13-ab61-4d07-90ac-8d3a0db00d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.713988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2c13-ab61-4d07-90ac-8d3a0db00d6d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.714280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2c13-ab61-4d07-90ac-8d3a0db00d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.714577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2c13-ab61-4d07-90ac-8d3a0db00d6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.715400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd2c13-ab61-4d07-90ac-8d3a0db00d6d None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.domain.id=default, user.id=f7156356453b494e8893e4adc4a9ad81) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:19.715719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd2c13-ab61-4d07-90ac-8d3a0db00d6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.717177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd2c13-ab61-4d07-90ac-8d3a0db00d6d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.725594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 28/56] 10.222.0.49 () {56 vars in 1322 bytes} [Tue Aug 2 10:00:19 2022] GET /identity/v3/role_assignments?user.id=f7156356453b494e8893e4adc4a9ad81&scope.domain.id=default&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:19.775756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 29/57] 10.222.0.49 () {60 vars in 1140 bytes} [Tue Aug 2 10:00:19 2022] POST /identity/v3/projects => generated 298 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:21.819898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374d197-9540-42d7-b070-4d5a20c9ec3f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.820483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374d197-9540-42d7-b070-4d5a20c9ec3f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.820483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374d197-9540-42d7-b070-4d5a20c9ec3f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.821800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 29/58] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:21.826976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331d572-019d-40be-aaf2-609bbfe21c22 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.827393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331d572-019d-40be-aaf2-609bbfe21c22 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.827595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331d572-019d-40be-aaf2-609bbfe21c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.845856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b331d572-019d-40be-aaf2-609bbfe21c22 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:21.847049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b331d572-019d-40be-aaf2-609bbfe21c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:21.911641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 30/59] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:21.935295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374d197-9540-42d7-b070-4d5a20c9ec3f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:21.948391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:22.034415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374d197-9540-42d7-b070-4d5a20c9ec3f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:22.066930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036e79e-9cfd-49b0-aee2-79cf6c4c0977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:22.067586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036e79e-9cfd-49b0-aee2-79cf6c4c0977 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:22.067871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036e79e-9cfd-49b0-aee2-79cf6c4c0977 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:22.068147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036e79e-9cfd-49b0-aee2-79cf6c4c0977 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:22.069306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036e79e-9cfd-49b0-aee2-79cf6c4c0977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:22.070803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036e79e-9cfd-49b0-aee2-79cf6c4c0977 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:22.072356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 30/60] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:22.077819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331d572-019d-40be-aaf2-609bbfe21c22 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:22.123014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:22.174204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331d572-019d-40be-aaf2-609bbfe21c22 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:22.207481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32210b0f-d221-491a-a29d-55bd9b51ad41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:22.208121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210b0f-d221-491a-a29d-55bd9b51ad41 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:22.208409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210b0f-d221-491a-a29d-55bd9b51ad41 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:22.208692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210b0f-d221-491a-a29d-55bd9b51ad41 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:22.209433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32210b0f-d221-491a-a29d-55bd9b51ad41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:22.210893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32210b0f-d221-491a-a29d-55bd9b51ad41 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:22.333754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 31/61] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:22 2022] POST /identity/v3/users => generated 275 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:25.496016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d2f28-e364-44bd-844d-26df0e86c445 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.496556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d2f28-e364-44bd-844d-26df0e86c445 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.496556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d2f28-e364-44bd-844d-26df0e86c445 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.498125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 31/62] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:25.503224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b42d0-1663-42de-b0d0-22fd357f242b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b42d0-1663-42de-b0d0-22fd357f242b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.503823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b42d0-1663-42de-b0d0-22fd357f242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.534217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-708b42d0-1663-42de-b0d0-22fd357f242b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:25.535474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-708b42d0-1663-42de-b0d0-22fd357f242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:25.595365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb116c-24dc-41f2-b575-fc42a384dc4a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.595697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb116c-24dc-41f2-b575-fc42a384dc4a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.595936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb116c-24dc-41f2-b575-fc42a384dc4a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.597308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 32/63] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:25.603148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5384df-0ae5-4783-9552-cd91ad86de15 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.603472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5384df-0ae5-4783-9552-cd91ad86de15 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.603762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5384df-0ae5-4783-9552-cd91ad86de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.630297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6e5384df-0ae5-4783-9552-cd91ad86de15 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:25.632121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6e5384df-0ae5-4783-9552-cd91ad86de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:25.661236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 32/64] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:25.702817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b42d0-1663-42de-b0d0-22fd357f242b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:25.751103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 33/65] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 02 10:00:25.752673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:25.760450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1072aa-190a-4756-a55a-e2ea86b5b1f6 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.760746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1072aa-190a-4756-a55a-e2ea86b5b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.764039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1072aa-190a-4756-a55a-e2ea86b5b1f6 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.764039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 34/66] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:25.768320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218b08-de21-41c5-9206-2404c9b4bbc0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.768675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218b08-de21-41c5-9206-2404c9b4bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.768966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218b08-de21-41c5-9206-2404c9b4bbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.795295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-76218b08-de21-41c5-9206-2404c9b4bbc0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:25.796512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-76218b08-de21-41c5-9206-2404c9b4bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:25.805319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:25.851996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b42d0-1663-42de-b0d0-22fd357f242b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:25.876377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 35/67] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:25.877376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76218b08-de21-41c5-9206-2404c9b4bbc0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:25.897358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957e1f6-bfeb-49e7-8bf8-1f91d600c81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:25.898348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957e1f6-bfeb-49e7-8bf8-1f91d600c81e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.898644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957e1f6-bfeb-49e7-8bf8-1f91d600c81e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.898932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957e1f6-bfeb-49e7-8bf8-1f91d600c81e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.912342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957e1f6-bfeb-49e7-8bf8-1f91d600c81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-5957e1f6-bfeb-49e7-8bf8-1f91d600c81e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:25.917358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:25.918183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:25.919492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 33/68] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:25.920741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043fc43-8078-4008-846f-906c284e4e69 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:25.920741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043fc43-8078-4008-846f-906c284e4e69 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:25.920854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043fc43-8078-4008-846f-906c284e4e69 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:25.922077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 34/69] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:25.923393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043fc43-8078-4008-846f-906c284e4e69 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:25.995408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76218b08-de21-41c5-9206-2404c9b4bbc0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.028799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.036625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.037319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.042444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.042444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.042641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043fc43-8078-4008-846f-906c284e4e69 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.045484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.049682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:26.055300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:26.056121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:26.056956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 36/70] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:26.058042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.081169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b3829-3614-4883-a8b8-0f0a11ed3c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.082090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b3829-3614-4883-a8b8-0f0a11ed3c8e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.082504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b3829-3614-4883-a8b8-0f0a11ed3c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.082829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b3829-3614-4883-a8b8-0f0a11ed3c8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.094688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b3829-3614-4883-a8b8-0f0a11ed3c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-200b3829-3614-4883-a8b8-0f0a11ed3c8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:26.099691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:26.100590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:26.101344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 35/71] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:00:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:26.102491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625f58-c134-4b07-92b4-e6cb3bca24cc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.102491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625f58-c134-4b07-92b4-e6cb3bca24cc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.102646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625f58-c134-4b07-92b4-e6cb3bca24cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.123320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-84625f58-c134-4b07-92b4-e6cb3bca24cc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:26.124534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-84625f58-c134-4b07-92b4-e6cb3bca24cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:26.168611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5ccd-f6a9-45f1-ae70-76d8b2895a39 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.196061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 36/72] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:26.196918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84625f58-c134-4b07-92b4-e6cb3bca24cc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.206512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.207151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.207467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.207876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.208606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:26.208921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.210471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.219818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 37/73] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:26.220628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.281498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.287982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.307137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84625f58-c134-4b07-92b4-e6cb3bca24cc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.327258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4260f-fbdc-492f-8ef7-66458d462c2a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.344144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39df27-3faf-4f8a-b096-ab3c2f81ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.344783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39df27-3faf-4f8a-b096-ab3c2f81ae2c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.345072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39df27-3faf-4f8a-b096-ab3c2f81ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.345357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39df27-3faf-4f8a-b096-ab3c2f81ae2c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.346088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39df27-3faf-4f8a-b096-ab3c2f81ae2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:26.346403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39df27-3faf-4f8a-b096-ab3c2f81ae2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.347859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39df27-3faf-4f8a-b096-ab3c2f81ae2c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.359707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 37/74] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:26.360955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33427-06a4-4fcc-b5ac-a2bd92c617df None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.361253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33427-06a4-4fcc-b5ac-a2bd92c617df None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.361538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33427-06a4-4fcc-b5ac-a2bd92c617df None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.366652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 38/75] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:26.366652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33427-06a4-4fcc-b5ac-a2bd92c617df None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.367332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.367941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.368224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.368505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.369239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:26.369546 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.370946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.379368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 38/76] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:26.380185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.474470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33427-06a4-4fcc-b5ac-a2bd92c617df None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.485898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0b39-d553-49e3-809e-3f626b910124 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.509455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.510765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97f6e9-a3e6-47bc-85e8-40a5ee85b044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.511179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97f6e9-a3e6-47bc-85e8-40a5ee85b044 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.511438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97f6e9-a3e6-47bc-85e8-40a5ee85b044 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.511721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97f6e9-a3e6-47bc-85e8-40a5ee85b044 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.520708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.521366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.521652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.524605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] PATH_INFO: `/v3/users/f7156356453b494e8893e4adc4a9ad81` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.524697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97f6e9-a3e6-47bc-85e8-40a5ee85b044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:26.524779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] RBAC: Authorizing `identity:get_user(user_id=f7156356453b494e8893e4adc4a9ad81)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.524779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.526413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 39/77] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/users/f7156356453b494e8893e4adc4a9ad81 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:26.529562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-bb97f6e9-a3e6-47bc-85e8-40a5ee85b044 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:26.529683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:26.530467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 39/78] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:26.531246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b4166-f446-4345-8092-01319043fd25 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.532413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b4166-f446-4345-8092-01319043fd25 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.532413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b4166-f446-4345-8092-01319043fd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.538422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.550365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-269b4166-f446-4345-8092-01319043fd25 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:26.551643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-269b4166-f446-4345-8092-01319043fd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:26.620657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 40/79] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:26.621500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b4166-f446-4345-8092-01319043fd25 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.638149 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4928a53-4abc-4cae-90cc-9938a89f4c93 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.671818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.672465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.672753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.673038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.686707 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.688279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.689938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 40/80] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:26.690719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.731498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b4166-f446-4345-8092-01319043fd25 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.738491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.760760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.765610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.766264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.766550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.766832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.768063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.769526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.771138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 41/81] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:26.772012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.796224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d4864-f390-46e9-aee7-f5abb0d10782 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.828048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.828678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.828968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.829260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.872165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8bd85-2298-40ea-ba4d-80200150020b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.895447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.896964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.905641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.906273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.906565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.906850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.907603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:26.907918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.909299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.919065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 42/82] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:26.919903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.944687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.981554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.994384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 41/83] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:26.995417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.020246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1a187-0445-42fc-8f43-e6a85c46de28 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.055906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.056553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.056842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.057122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.058272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.059761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.061546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 43/84] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:27.065981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.104454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc56d7-993d-4600-bc9e-2524298b6837 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.138141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.138770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.139058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.139371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.156671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:27.167842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb151e6a-49e6-4598-ac8f-24c090759c3c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.197669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:27.203480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.204110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.204407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.204686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.206518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.208034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.271802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.273281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.305119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 42/85] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:27.305985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.354400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:27.373176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 44/86] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:27.374275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.404172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:27.412842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb5cb0-42eb-4fc4-8748-3925d0549cdc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.448668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.451948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.451948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.451948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.451948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.452525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.454159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 43/87] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:27.454927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.478861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bd042-e304-498a-a9b7-216e1b5a5ca1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.513349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.514014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.514317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.514599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.515815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.517312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.518990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 45/88] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:27.519781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.565927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7208742-b75f-41d8-a3d6-af338b36ff50 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.582804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:27.601360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.601990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.602283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.602572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.603297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.604760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.617895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:27.621730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56c828-d2d3-4743-9260-788214fde184 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.655693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.656390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.656696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.656997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.658246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.659774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.661458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 46/89] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:27.662213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.756241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831119c-60b5-4f2e-9d59-0e54ae6607ac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.789744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.790175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 44/90] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:27 2022] POST /identity/v3/users => generated 289 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:27.790397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.790691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.790976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.791532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.792336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.793812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.795850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 47/91] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:27.800194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:27.844669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:27.919143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1685-437d-400c-847d-7ebee99cd87f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.931616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f2c93-e958-4e3f-b2d9-b720240cda7d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:27.962783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.963472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.963767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.964049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.971647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:27.973298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:27.973586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:27.973866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:27.974675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=f7156356453b494e8893e4adc4a9ad81) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:27.975024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:27.976514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:27.985474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 48/92] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/role_assignments?user.id=f7156356453b494e8893e4adc4a9ad81&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:27.986749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:28.009570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:28.050962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:28.053177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:28.130664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:28.136065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93add9d-e664-4855-8d80-a46133835428 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:28.187797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:28.188436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:28.188726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:28.189001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:28.189807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=bd58a6d89a6c4fa196454cb9055bbb5c) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:28.190123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:28.191619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:28.191726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 45/93] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:28.192567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:28.198909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 49/94] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:27 2022] GET /identity/v3/role_assignments?user.id=bd58a6d89a6c4fa196454cb9055bbb5c&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:28.201759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:28.283122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:28.319865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f3f7b-bb4f-4817-a613-b9e483ec68a5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:28.322090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379c481-48bd-41ea-b6a2-2d17b307126f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:28.360194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:28.361226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:28.361714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:28.362197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:28.363669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] RBAC: Adding query filter params (role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=bd58a6d89a6c4fa196454cb9055bbb5c) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:28.364224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:28.366647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:28.379325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 46/95] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:28 2022] GET /identity/v3/role_assignments?user.id=bd58a6d89a6c4fa196454cb9055bbb5c&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:28.382574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921b5fd-e9d4-47b3-a662-95235e2fdc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:28.383522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921b5fd-e9d4-47b3-a662-95235e2fdc3d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:28.383979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921b5fd-e9d4-47b3-a662-95235e2fdc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:28.384412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921b5fd-e9d4-47b3-a662-95235e2fdc3d None admin] PATH_INFO: `/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:28.386064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921b5fd-e9d4-47b3-a662-95235e2fdc3d None admin] RBAC: Authorizing `identity:get_project(project_id=6041ae73e5ac43bca8d4eb1bef06515b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:28.388060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921b5fd-e9d4-47b3-a662-95235e2fdc3d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:28.390290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 50/96] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:28 2022] GET /identity/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:28.397917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:28.523600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466146b-bb89-4df8-8b82-5a3623121a95 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:28.562420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75228c0b-41c3-460b-a235-2d64994d1a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:28.563132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228c0b-41c3-460b-a235-2d64994d1a86 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:28.563560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228c0b-41c3-460b-a235-2d64994d1a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:28.563846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228c0b-41c3-460b-a235-2d64994d1a86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:28.564656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75228c0b-41c3-460b-a235-2d64994d1a86 None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=6041ae73e5ac43bca8d4eb1bef06515b, user.id=bd58a6d89a6c4fa196454cb9055bbb5c) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:28.564976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75228c0b-41c3-460b-a235-2d64994d1a86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:28.566526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75228c0b-41c3-460b-a235-2d64994d1a86 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:28.570446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:28.573740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 47/97] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:28 2022] GET /identity/v3/role_assignments?user.id=bd58a6d89a6c4fa196454cb9055bbb5c&scope.project.id=6041ae73e5ac43bca8d4eb1bef06515b&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:30.393989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d6ed-e4cf-4d54-948f-51ce132403c7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.394998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d6ed-e4cf-4d54-948f-51ce132403c7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.395523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d6ed-e4cf-4d54-948f-51ce132403c7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.397154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 51/98] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:30.402675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a02205-4fc2-419b-8254-21810a14ecad None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.403070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a02205-4fc2-419b-8254-21810a14ecad None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.403302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a02205-4fc2-419b-8254-21810a14ecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.426193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-74a02205-4fc2-419b-8254-21810a14ecad None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:30.427510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-74a02205-4fc2-419b-8254-21810a14ecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:30.503105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 48/99] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:30.526340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d6ed-e4cf-4d54-948f-51ce132403c7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:30.576817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:30.672432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d6ed-e4cf-4d54-948f-51ce132403c7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:30.718390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0047e42-9cfb-451f-bcd7-669e3edbebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:30.719053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0047e42-9cfb-451f-bcd7-669e3edbebd1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.719364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0047e42-9cfb-451f-bcd7-669e3edbebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.719643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0047e42-9cfb-451f-bcd7-669e3edbebd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.740140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0047e42-9cfb-451f-bcd7-669e3edbebd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a0047e42-9cfb-451f-bcd7-669e3edbebd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:30.746726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:30.749194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:30.749982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 52/100] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:30.756779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a02205-4fc2-419b-8254-21810a14ecad None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:30.798898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef7eac-c62d-40fd-8cf0-e314d1891035 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.798898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef7eac-c62d-40fd-8cf0-e314d1891035 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.798898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef7eac-c62d-40fd-8cf0-e314d1891035 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.798898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 53/101] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:30.803613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b08b6f-3c57-4b14-95cd-58bf6095c759 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.803851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b08b6f-3c57-4b14-95cd-58bf6095c759 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.804135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b08b6f-3c57-4b14-95cd-58bf6095c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.839055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a6b08b6f-3c57-4b14-95cd-58bf6095c759 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:30.840292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a6b08b6f-3c57-4b14-95cd-58bf6095c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:30.890179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a02205-4fc2-419b-8254-21810a14ecad None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:30.931710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:30.932734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:30.933320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:30.933795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:30.934961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:30.935503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:30.937187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:30.950737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 49/102] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:30.955247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:30.957293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:30.974583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 54/103] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 02 10:00:31.005378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b08b6f-3c57-4b14-95cd-58bf6095c759 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.040445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:31.113183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6029d-a85e-460f-b5d5-c56732c75124 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.155213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.155213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.155213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.155213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] PATH_INFO: `/v3/users/f7156356453b494e8893e4adc4a9ad81` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.155605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] RBAC: Authorizing `identity:get_user(user_id=f7156356453b494e8893e4adc4a9ad81)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:31.155605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:31.156267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 50/104] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:30 2022] GET /identity/v3/users/f7156356453b494e8893e4adc4a9ad81 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:31.161583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.222157 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b08b6f-3c57-4b14-95cd-58bf6095c759 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.229087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:31.279348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abffc74-8159-4b86-adf5-e43a3bb18670 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.301341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.302361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.302870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.303415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.317483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea26e24-c221-4b4f-a875-532ab5fb73df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.318146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26e24-c221-4b4f-a875-532ab5fb73df None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.318441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26e24-c221-4b4f-a875-532ab5fb73df None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.318762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26e24-c221-4b4f-a875-532ab5fb73df None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.319975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea26e24-c221-4b4f-a875-532ab5fb73df None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:31.321517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea26e24-c221-4b4f-a875-532ab5fb73df None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:31.323239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 51/105] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:31 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:31.324387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f91ff-b9aa-4b64-b039-82e579db72bf None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.324681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f91ff-b9aa-4b64-b039-82e579db72bf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.324967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f91ff-b9aa-4b64-b039-82e579db72bf None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.326318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 52/106] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:31.331611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910193f-f03a-4d22-a421-54c0f2c8995d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.331910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910193f-f03a-4d22-a421-54c0f2c8995d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.332199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910193f-f03a-4d22-a421-54c0f2c8995d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.339024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:31.344439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:31.345282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:31.346041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:31.348375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 55/107] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:31.349329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.355207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7910193f-f03a-4d22-a421-54c0f2c8995d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:31.355207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7910193f-f03a-4d22-a421-54c0f2c8995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:31.429081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 53/108] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:31.429958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910193f-f03a-4d22-a421-54c0f2c8995d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.515836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:31.561699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910193f-f03a-4d22-a421-54c0f2c8995d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.567562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bcaaa-f1fd-421b-8bed-47caa1865d19 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.583596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:31.600358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.601333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.601767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.602374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.603274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:31.603732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:31.603881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a9ba3-353a-4588-be78-d79f5444d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.604639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9ba3-353a-4588-be78-d79f5444d0a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.604992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9ba3-353a-4588-be78-d79f5444d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.605335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9ba3-353a-4588-be78-d79f5444d0a9 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/f7156356453b494e8893e4adc4a9ad81/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.605920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:31.607513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a9ba3-353a-4588-be78-d79f5444d0a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=37f4bbc3aa894e06971fb36f16623f07, user_id=f7156356453b494e8893e4adc4a9ad81, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:31.608989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a9ba3-353a-4588-be78-d79f5444d0a9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:31.616636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 54/109] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:31.617748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.703386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 56/110] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:31 2022] PUT /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/f7156356453b494e8893e4adc4a9ad81/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:31.705237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ea15e-0aaf-46db-901d-7c24040036fd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.705666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ea15e-0aaf-46db-901d-7c24040036fd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.705959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ea15e-0aaf-46db-901d-7c24040036fd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.707729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 57/111] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:31.708374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ea15e-0aaf-46db-901d-7c24040036fd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.732036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19550-0f1a-4bf7-a01e-78f5a41b368c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.756973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:31.772022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0f93a-bc3c-45d0-b8ce-d2b94a45e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.772768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0f93a-bc3c-45d0-b8ce-d2b94a45e6fb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.773129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0f93a-bc3c-45d0-b8ce-d2b94a45e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.773482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0f93a-bc3c-45d0-b8ce-d2b94a45e6fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.786642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0f93a-bc3c-45d0-b8ce-d2b94a45e6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fee0f93a-bc3c-45d0-b8ce-d2b94a45e6fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:31.792908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:31.794035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:31.794989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:31.800957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 55/112] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:31.800957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113d439-0e69-4512-a822-4ce27fa45ac9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.800957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113d439-0e69-4512-a822-4ce27fa45ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.800957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113d439-0e69-4512-a822-4ce27fa45ac9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.804688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 56/113] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:31.806083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18a79e-0666-4120-8c50-4bf1aa2d56bd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.806380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18a79e-0666-4120-8c50-4bf1aa2d56bd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.806664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18a79e-0666-4120-8c50-4bf1aa2d56bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.826638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1a18a79e-0666-4120-8c50-4bf1aa2d56bd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:31.828278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1a18a79e-0666-4120-8c50-4bf1aa2d56bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:31.861463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ea15e-0aaf-46db-901d-7c24040036fd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:31.909598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 57/114] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:31.910978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18a79e-0666-4120-8c50-4bf1aa2d56bd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:31.914306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f7064-a347-43dc-8163-40df3ab1a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:31.916927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7064-a347-43dc-8163-40df3ab1a369 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.916927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7064-a347-43dc-8163-40df3ab1a369 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.916927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7064-a347-43dc-8163-40df3ab1a369 None admin] PATH_INFO: `/v3/users/68db2bf12b0b4542846578ee2b7cced5` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.933251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:31.938240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f7064-a347-43dc-8163-40df3ab1a369 None admin] RBAC: Authorizing `identity:get_user(user_id=68db2bf12b0b4542846578ee2b7cced5)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:31.940362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f7064-a347-43dc-8163-40df3ab1a369 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:31.941703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 58/115] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:31 2022] GET /identity/v3/users/68db2bf12b0b4542846578ee2b7cced5 => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:31.942809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831abc62-50ab-463d-8e8d-d7d00cbc3b99 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:31.947555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831abc62-50ab-463d-8e8d-d7d00cbc3b99 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:31.947555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831abc62-50ab-463d-8e8d-d7d00cbc3b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:31.964335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-831abc62-50ab-463d-8e8d-d7d00cbc3b99 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:31.966651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-831abc62-50ab-463d-8e8d-d7d00cbc3b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:32.019666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18a79e-0666-4120-8c50-4bf1aa2d56bd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.031925 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:32.040129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 59/116] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:32.041421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bd86-8610-49e5-8439-ef102ea05f7b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.041718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bd86-8610-49e5-8439-ef102ea05f7b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.042001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bd86-8610-49e5-8439-ef102ea05f7b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.047287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 60/117] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:32.047287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bd86-8610-49e5-8439-ef102ea05f7b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.064234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.065120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.065517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.066159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.066925 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:32.067600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.069854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.078839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 58/118] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:32.084180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.161771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bd86-8610-49e5-8439-ef102ea05f7b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.189278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:32.194447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7b77-5aa2-4483-8554-9e5d7fd02b19 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.198661 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8decd32-a84c-4a5e-b603-5e061398c0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.203015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8decd32-a84c-4a5e-b603-5e061398c0cb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.203015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8decd32-a84c-4a5e-b603-5e061398c0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.203015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8decd32-a84c-4a5e-b603-5e061398c0cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.203015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8decd32-a84c-4a5e-b603-5e061398c0cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.203015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8decd32-a84c-4a5e-b603-5e061398c0cb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.209255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 61/119] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:32.209255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b1464-2d58-467f-bbe8-e9a74bab6280 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.209255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b1464-2d58-467f-bbe8-e9a74bab6280 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.209255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b1464-2d58-467f-bbe8-e9a74bab6280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.224895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-530b1464-2d58-467f-bbe8-e9a74bab6280 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:32.226165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-530b1464-2d58-467f-bbe8-e9a74bab6280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:32.239225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.239225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.239225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.239225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.280881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:32.306010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 62/120] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:32.307124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b1464-2d58-467f-bbe8-e9a74bab6280 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.331417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.332949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.415082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b1464-2d58-467f-bbe8-e9a74bab6280 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.442140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:32.443620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 59/121] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:32.444983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.458144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.459092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.464560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.464560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.472862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:32.478648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:32.479703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:32.480878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:32.481498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 63/122] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:32.482332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.536998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:32.563019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abac2d-80a7-456d-b595-ddca5d8f3f67 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.593210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44083026-a3b0-463b-83eb-fdb56e30c58d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.601750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.602501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.602848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.603131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.604447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.606030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.607969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 60/123] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:32.608539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.632354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.632996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.633442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.633731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.634424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.635923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.710930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43abf24-eff6-4e1c-9ca0-9e2871c9a037 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.716155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 64/124] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:00:32 2022] POST /identity/v3/projects => generated 302 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:32.716984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.757141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.757911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.758205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.758490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.771678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:32.772742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:32.778971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:32.786367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:32.787173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 61/125] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:32.790102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.795555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:32.852506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c393-58ae-4df9-8206-2d69f074c97c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.902194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.903352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.904086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.904575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.906414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.908242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.910441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 65/126] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:32.911313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:32.928664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8ecf-ed44-4545-b94c-1e30fbcb914a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:32.973512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:32.974434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:32.974703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:32.975118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:32.975992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:32.976306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:32.977711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:32.991279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 62/127] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:32.991279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.072474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:33.079341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1138-c558-452d-beb7-66fc58744ed9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.089180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:33.120761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.121419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.121714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.121995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.123119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b0b4-525a-41e9-8e67-03e97da3df1b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.123562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.125109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.127302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 66/128] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:33.128258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.173185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.174178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.175001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.175483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.176801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:33.177564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.180023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.191931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 63/129] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:00:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:33.193347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.239031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebdbcf-e89e-405a-8a7f-eb7322bcc118 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.276911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.277646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.278252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.278356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.279500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] RBAC: Adding query filter params (role.id=0d0f0d7ac63143048ef3044aa9650882, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=68db2bf12b0b4542846578ee2b7cced5) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:33.279886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.281550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.290434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 67/130] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:33 2022] GET /identity/v3/role_assignments?user.id=68db2bf12b0b4542846578ee2b7cced5&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=0d0f0d7ac63143048ef3044aa9650882 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:33.291601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.330725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b2785-de78-4614-a325-aaa9e7e060ad None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.338545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:33.368980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:33.380966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.382634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.383105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.383932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.385629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.388530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.390745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 64/131] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:33 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:33.391863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.452259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05972b8-edca-46f7-86dd-19bda71a8924 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.517327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.517327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.517327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.518411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/bd58a6d89a6c4fa196454cb9055bbb5c/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.524745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] RBAC: Authorizing `identity:create_grant(project_id=37f4bbc3aa894e06971fb36f16623f07, user_id=bd58a6d89a6c4fa196454cb9055bbb5c, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.526464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.566769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cc77e-42bc-4dbc-a7c5-775f6bed70e9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.603507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.604167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.604470 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.604729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.605918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.607716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.609745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 65/132] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:33 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:33.610648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.654059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 68/133] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:33 2022] PUT /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/bd58a6d89a6c4fa196454cb9055bbb5c/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:33.654749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.681567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:33.726967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1c82-43e1-4043-82e0-e76ac194ede6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.766194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.766866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.767166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.767520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] PATH_INFO: `/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.768788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] RBAC: Authorizing `identity:get_project(project_id=6041ae73e5ac43bca8d4eb1bef06515b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.770341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 66/134] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:33 2022] GET /identity/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:33.778401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.880637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:33.889725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0831c3-a573-4c66-bf59-2d2f3814a7cc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.959576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.960188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.960477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.960760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.963245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.964831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.966915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 69/135] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:33 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:33.974693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.996775 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325319e0-2494-4fe0-8adf-9dd9a207f286 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:34.060966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:34.074373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f1113-045f-48fd-8813-95183a675712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:34.075068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f1113-045f-48fd-8813-95183a675712 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.075380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f1113-045f-48fd-8813-95183a675712 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.082746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f1113-045f-48fd-8813-95183a675712 None admin] PATH_INFO: `/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b/users/bd58a6d89a6c4fa196454cb9055bbb5c/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.082746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f1113-045f-48fd-8813-95183a675712 None admin] RBAC: Authorizing `identity:create_grant(project_id=6041ae73e5ac43bca8d4eb1bef06515b, user_id=bd58a6d89a6c4fa196454cb9055bbb5c, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:34.082746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f1113-045f-48fd-8813-95183a675712 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:34.134302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3db6c-1f60-4646-a6da-18a556db7f3a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:34.175557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 67/136] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:33 2022] PUT /identity/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b/users/bd58a6d89a6c4fa196454cb9055bbb5c/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:34.193775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf146f-9454-4d2e-bdb4-7dda8f930f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:34.194579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf146f-9454-4d2e-bdb4-7dda8f930f28 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.194884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf146f-9454-4d2e-bdb4-7dda8f930f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.195166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf146f-9454-4d2e-bdb4-7dda8f930f28 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/bd58a6d89a6c4fa196454cb9055bbb5c/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.201469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf146f-9454-4d2e-bdb4-7dda8f930f28 None admin] RBAC: Authorizing `identity:create_grant(project_id=37f4bbc3aa894e06971fb36f16623f07, user_id=bd58a6d89a6c4fa196454cb9055bbb5c, role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:34.203224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf146f-9454-4d2e-bdb4-7dda8f930f28 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:34.281665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5d5f1-87ad-4782-8296-6604d91076ec None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.281855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 70/137] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:33 2022] PUT /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/bd58a6d89a6c4fa196454cb9055bbb5c/roles/9a49213a53964452a411e2dbd74d7c5e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:34.282410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5d5f1-87ad-4782-8296-6604d91076ec None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.282885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5d5f1-87ad-4782-8296-6604d91076ec None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.284526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 68/138] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:34.291477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b599e4a-029f-411b-9f85-b0349b58cecb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.292138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b599e4a-029f-411b-9f85-b0349b58cecb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.292716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b599e4a-029f-411b-9f85-b0349b58cecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.315782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5b599e4a-029f-411b-9f85-b0349b58cecb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:34.317492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5b599e4a-029f-411b-9f85-b0349b58cecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:34.349470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:34.410105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 71/139] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:34.436412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5d5f1-87ad-4782-8296-6604d91076ec None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:34.573750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5d5f1-87ad-4782-8296-6604d91076ec None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:34.613689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:34.614355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.614643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.614924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.627508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:34.632603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:34.633487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:34.634731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 69/140] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:34.640355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b599e4a-029f-411b-9f85-b0349b58cecb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:34.751433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b599e4a-029f-411b-9f85-b0349b58cecb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:34.790290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:34.790926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.791231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.791516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.792267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:34.792572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:34.794091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:34.803649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 72/141] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:34.809329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:34.868642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:34.927930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1a8d-2d89-4488-b506-bdeff545cf92 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:34.967603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:34.968260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:34.968545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:34.968818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] PATH_INFO: `/v3/users/f7156356453b494e8893e4adc4a9ad81` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:34.970056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] RBAC: Authorizing `identity:get_user(user_id=f7156356453b494e8893e4adc4a9ad81)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:34.971549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:34.973367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 70/142] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:34 2022] GET /identity/v3/users/f7156356453b494e8893e4adc4a9ad81 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:34.978924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.007236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:35.108391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69e680-3823-4669-b1a6-f4cec419d3ae None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.157554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7a144-68a0-40ac-a6b4-2cdfd8d77bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.158169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a144-68a0-40ac-a6b4-2cdfd8d77bfb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.158463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a144-68a0-40ac-a6b4-2cdfd8d77bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.158743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a144-68a0-40ac-a6b4-2cdfd8d77bfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.274670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7a144-68a0-40ac-a6b4-2cdfd8d77bfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.276235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7a144-68a0-40ac-a6b4-2cdfd8d77bfb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.300331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:35.430086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 73/143] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:35.435548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.558746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a780a-6ab5-459a-8773-5f4e5b2ab95e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.572733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b106c-f8d3-47bd-bc84-71f00ba96c53 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.573043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b106c-f8d3-47bd-bc84-71f00ba96c53 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.573294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b106c-f8d3-47bd-bc84-71f00ba96c53 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.574771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 74/144] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:35.580981 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:35.592914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffbd5a-76be-4ad2-87c5-6456fcb12767 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.595660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffbd5a-76be-4ad2-87c5-6456fcb12767 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.595660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffbd5a-76be-4ad2-87c5-6456fcb12767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.622039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b2ffbd5a-76be-4ad2-87c5-6456fcb12767 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:35.624600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.625291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.625611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.625902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.627826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b2ffbd5a-76be-4ad2-87c5-6456fcb12767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:35.630535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.632652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.634944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 71/145] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:35 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:35.642373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.777296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2975f66-5667-43bc-8728-b6fd0eb9b4fd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.805716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 75/146] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:35.822256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9185d-a52a-492c-8e74-ad9b4370636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.822896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9185d-a52a-492c-8e74-ad9b4370636b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.823208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9185d-a52a-492c-8e74-ad9b4370636b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.823601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9185d-a52a-492c-8e74-ad9b4370636b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.824409 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9185d-a52a-492c-8e74-ad9b4370636b None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=f7156356453b494e8893e4adc4a9ad81) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:35.824721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9185d-a52a-492c-8e74-ad9b4370636b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.826236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9185d-a52a-492c-8e74-ad9b4370636b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.833278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 72/147] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:35 2022] GET /identity/v3/role_assignments?user.id=f7156356453b494e8893e4adc4a9ad81&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:35.852695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffbd5a-76be-4ad2-87c5-6456fcb12767 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.925456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:35.993774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffbd5a-76be-4ad2-87c5-6456fcb12767 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.046406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b556ec-8a4c-42d3-8d3a-0ab96c806fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.047214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b556ec-8a4c-42d3-8d3a-0ab96c806fcd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.047512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b556ec-8a4c-42d3-8d3a-0ab96c806fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.047791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b556ec-8a4c-42d3-8d3a-0ab96c806fcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.049366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b556ec-8a4c-42d3-8d3a-0ab96c806fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.049777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674ca3a-e186-42a1-be42-a1c4da377b1f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.050071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674ca3a-e186-42a1-be42-a1c4da377b1f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.050351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674ca3a-e186-42a1-be42-a1c4da377b1f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.050841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b556ec-8a4c-42d3-8d3a-0ab96c806fcd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.051810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 73/148] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:36.053655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 76/149] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:36.059551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674ca3a-e186-42a1-be42-a1c4da377b1f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.066603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5a101-62d6-4d8c-a53a-7c5f1acfdc0e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.066884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5a101-62d6-4d8c-a53a-7c5f1acfdc0e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.067169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5a101-62d6-4d8c-a53a-7c5f1acfdc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.109054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-77d5a101-62d6-4d8c-a53a-7c5f1acfdc0e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:36.110277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-77d5a101-62d6-4d8c-a53a-7c5f1acfdc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:36.188830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:36.269057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 77/150] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:36.282429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674ca3a-e186-42a1-be42-a1c4da377b1f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.299521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5a101-62d6-4d8c-a53a-7c5f1acfdc0e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.355770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.356424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.356747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.357129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.357910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.359476 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.388022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:36.454857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5a101-62d6-4d8c-a53a-7c5f1acfdc0e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.500068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.500743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.501031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.501312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.517394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:36.524124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 74/151] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/users => generated 283 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:36.532847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:36.533651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:36.534384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 78/152] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:36.541444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.707774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a78c5d-f98a-488f-8300-6169f9c32aaf None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.711169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:36.750724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173dda2-7def-424c-ac84-f257ce2cc66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.751400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173dda2-7def-424c-ac84-f257ce2cc66a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.751684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173dda2-7def-424c-ac84-f257ce2cc66a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.751959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173dda2-7def-424c-ac84-f257ce2cc66a None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.752685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173dda2-7def-424c-ac84-f257ce2cc66a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:36.752990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173dda2-7def-424c-ac84-f257ce2cc66a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.754482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173dda2-7def-424c-ac84-f257ce2cc66a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.764647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 75/153] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:36.769930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.886389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7072-aeb6-444d-9a94-01b54425b611 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.886730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7072-aeb6-444d-9a94-01b54425b611 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.886949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7072-aeb6-444d-9a94-01b54425b611 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.888726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 76/154] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:36.896417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f60d5-e20d-44e2-a81d-97a42894a421 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.897725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f60d5-e20d-44e2-a81d-97a42894a421 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.898206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f60d5-e20d-44e2-a81d-97a42894a421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.899268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57345a6-1c84-4856-a8ab-a00ce99d8553 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.907651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:36.919623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cd8f60d5-e20d-44e2-a81d-97a42894a421 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:36.920934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cd8f60d5-e20d-44e2-a81d-97a42894a421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:36.950921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.951973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.952499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.952968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] PATH_INFO: `/v3/users/68db2bf12b0b4542846578ee2b7cced5` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.954561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] RBAC: Authorizing `identity:get_user(user_id=68db2bf12b0b4542846578ee2b7cced5)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.956470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.958695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 79/155] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/users/68db2bf12b0b4542846578ee2b7cced5 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:36.964865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.995929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 77/156] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:37.026078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f60d5-e20d-44e2-a81d-97a42894a421 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.089009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9182c2-ce68-4d38-992f-6288f201555a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.142498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.143555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.144002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.144500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.145849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:37.148598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:37.151031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 80/157] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:37.157087 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.165196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:37.181939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f60d5-e20d-44e2-a81d-97a42894a421 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.228678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:37.232556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3110c0-d8a6-4b66-b505-3282abae0f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.233236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3110c0-d8a6-4b66-b505-3282abae0f0b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.233525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3110c0-d8a6-4b66-b505-3282abae0f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.233808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3110c0-d8a6-4b66-b505-3282abae0f0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.247795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3110c0-d8a6-4b66-b505-3282abae0f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8a3110c0-d8a6-4b66-b505-3282abae0f0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:37.252889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:37.253964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:37.254724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 78/158] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:37.255479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb01bb-c9ca-48d7-ba2d-5d1e39856e12 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.255973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb01bb-c9ca-48d7-ba2d-5d1e39856e12 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.255973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb01bb-c9ca-48d7-ba2d-5d1e39856e12 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.258206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 79/159] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:37.261452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb01bb-c9ca-48d7-ba2d-5d1e39856e12 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.309230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd218c8f-ea42-44ac-9877-e2c7c1f50b6a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.350103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be9db-d91c-400f-b6d8-6387fa35c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.350895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be9db-d91c-400f-b6d8-6387fa35c323 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.351182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be9db-d91c-400f-b6d8-6387fa35c323 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.352079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be9db-d91c-400f-b6d8-6387fa35c323 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/68db2bf12b0b4542846578ee2b7cced5/roles/0d0f0d7ac63143048ef3044aa9650882` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.354224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be9db-d91c-400f-b6d8-6387fa35c323 None admin] RBAC: Authorizing `identity:create_grant(project_id=37f4bbc3aa894e06971fb36f16623f07, user_id=68db2bf12b0b4542846578ee2b7cced5, role_id=0d0f0d7ac63143048ef3044aa9650882)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:37.355766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be9db-d91c-400f-b6d8-6387fa35c323 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:37.410525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb01bb-c9ca-48d7-ba2d-5d1e39856e12 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.436328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 81/160] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:37 2022] PUT /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07/users/68db2bf12b0b4542846578ee2b7cced5/roles/0d0f0d7ac63143048ef3044aa9650882 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:37.437551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b9e2d-d8b5-4704-997e-a899abd0b68e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.437855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b9e2d-d8b5-4704-997e-a899abd0b68e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.438240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b9e2d-d8b5-4704-997e-a899abd0b68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.478446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0d8b9e2d-d8b5-4704-997e-a899abd0b68e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:37.480286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0d8b9e2d-d8b5-4704-997e-a899abd0b68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:37.486838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.487513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.487807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.488089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.489090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:37.489402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:37.491393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:37.514384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 80/161] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:37.523837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.556297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:37.601095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 82/162] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:37.606286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2172456-b54a-4019-beb4-05fa2b89376d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.608265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2172456-b54a-4019-beb4-05fa2b89376d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.608785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2172456-b54a-4019-beb4-05fa2b89376d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.611290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 83/163] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:37.620368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561ff0-1235-4479-934a-e791e8fe2cdc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.620984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561ff0-1235-4479-934a-e791e8fe2cdc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.621488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561ff0-1235-4479-934a-e791e8fe2cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.651046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-46561ff0-1235-4479-934a-e791e8fe2cdc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:37.653451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-46561ff0-1235-4479-934a-e791e8fe2cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:37.697061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:37.723388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea6a6-32ac-4890-be32-11151c58ca39 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.743012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 84/164] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 02 10:00:37.743725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46561ff0-1235-4479-934a-e791e8fe2cdc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.798205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.799253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.799735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.800230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.801652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:37.807527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:37.809953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 81/165] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:37.811034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.876703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46561ff0-1235-4479-934a-e791e8fe2cdc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.918434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.920101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.920383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.920657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.939408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:37.950232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:37.951137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:37.951930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:37.954179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 85/166] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:37.955042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.000110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.043481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.074729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12214002-a59e-419d-b717-6e492bdf9fcd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.085515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d00e6-65da-440a-ba13-2b3dad783a04 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.125300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.125943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.126560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.148884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.149937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.150416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.150880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.184193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:38.189548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:38.190417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:38.191208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:38.193735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 82/167] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:38.198747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.208821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.210892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.241591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.331811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 86/168] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:38.332395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.490726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca87d9-0d19-48af-8b3c-4e1f8d25d44a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.537730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a63e22-6c9e-4204-bb2a-0e687ab05d6a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.553119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.587959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.588514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.588804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.589082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.589544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.589818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:38.590126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.590797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.591370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.591642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.592058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.592985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:38.593490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.595056 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.601251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 83/169] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:38.601912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.612603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 87/170] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:38.614101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.616254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.812505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b485d-f8db-4f80-9294-4fe921d4d1af None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.850235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.851145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.851600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.852035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.853580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.853770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97509b2b-fa75-4e35-b447-b7ffc8968222 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:38.855873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.857691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 88/171] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:38.858766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.918935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:38.948966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:38.949626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:38.950097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:38.950518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:38.955981 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:38.957474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:38.959423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 84/172] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:38.964046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:38.977118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7bcdb-6134-461d-b373-6e6fb8defca6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:39.021387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:39.022086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:39.022371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:39.022647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] PATH_INFO: `/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:39.023896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] RBAC: Authorizing `identity:get_user(user_id=bd58a6d89a6c4fa196454cb9055bbb5c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:39.025417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:39.027091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 89/173] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/users/bd58a6d89a6c4fa196454cb9055bbb5c => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:39.027890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:39.137506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159c399-4934-45d9-b0d3-d423a8d6edb4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:39.157529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:39.160355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:39.176301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:39.176943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:39.177225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:39.177505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:39.178434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=bd58a6d89a6c4fa196454cb9055bbb5c) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:39.178751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:39.180288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:39.187010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 90/174] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:39 2022] GET /identity/v3/role_assignments?user.id=bd58a6d89a6c4fa196454cb9055bbb5c&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:39.187699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:39.242045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ba94-a00d-4b2a-9d5d-0f01b758f8d3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:39.316258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f799da-38e8-4252-8077-2252726c4635 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:39.341310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:39.341958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:39.342243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:39.342520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:39.362996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:39.363986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:39.364272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:39.364547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:39.464528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:39.481203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:39.491081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:39.521390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:39.523933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:39.600780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 91/175] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:39.607879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:39.711053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:39.768880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 85/176] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 805 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:39.775021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:39.793214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:39.823903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ee633-216d-492a-898a-25cad0e8a98b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:39.909028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:39.909691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:39.909974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:39.910248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] PATH_INFO: `/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:39.914602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] RBAC: Authorizing `identity:get_project(project_id=6041ae73e5ac43bca8d4eb1bef06515b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:39.918423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:39.923158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 92/177] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:39 2022] GET /identity/v3/projects/6041ae73e5ac43bca8d4eb1bef06515b => generated 312 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:39.938335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:39.982512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efabe33-a361-4d73-a7aa-e95ebdff9674 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:40.095864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:40.096441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.096727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.097008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.104668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:40.106172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:40.110723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 86/178] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:39 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:40.117768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:40.163684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09b048-fb5c-4e26-887a-5e55f66ed351 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:40.201127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:40.219918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee157a-f15d-41d5-b3f5-fcdda6466f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:40.222430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee157a-f15d-41d5-b3f5-fcdda6466f7e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.222430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee157a-f15d-41d5-b3f5-fcdda6466f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.222430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee157a-f15d-41d5-b3f5-fcdda6466f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.222430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee157a-f15d-41d5-b3f5-fcdda6466f7e None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=6041ae73e5ac43bca8d4eb1bef06515b, user.id=bd58a6d89a6c4fa196454cb9055bbb5c) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:40.222893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee157a-f15d-41d5-b3f5-fcdda6466f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:40.224545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee157a-f15d-41d5-b3f5-fcdda6466f7e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:40.231116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 93/179] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:39 2022] GET /identity/v3/role_assignments?user.id=bd58a6d89a6c4fa196454cb9055bbb5c&scope.project.id=6041ae73e5ac43bca8d4eb1bef06515b&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:40.232543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8db3f-6206-4faa-96d4-3fe1116767f7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.233015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8db3f-6206-4faa-96d4-3fe1116767f7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.239233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8db3f-6206-4faa-96d4-3fe1116767f7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.241117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 94/180] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:40.252713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af76e0-2e09-42ee-be09-6d08f4b8b30b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.253281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af76e0-2e09-42ee-be09-6d08f4b8b30b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.253762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af76e0-2e09-42ee-be09-6d08f4b8b30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.274676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf9116-1c38-4d09-bb3e-efbb923da988 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:40.292288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-04af76e0-2e09-42ee-be09-6d08f4b8b30b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:40.293787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-04af76e0-2e09-42ee-be09-6d08f4b8b30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:40.315441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25543c82-464e-4145-b5d3-c8a8f9b3ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:40.316085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25543c82-464e-4145-b5d3-c8a8f9b3ea9c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.316374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25543c82-464e-4145-b5d3-c8a8f9b3ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.316654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25543c82-464e-4145-b5d3-c8a8f9b3ea9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.318120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:40.319486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25543c82-464e-4145-b5d3-c8a8f9b3ea9c None admin] RBAC: Adding query filter params (role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=bd58a6d89a6c4fa196454cb9055bbb5c) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:40.319857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25543c82-464e-4145-b5d3-c8a8f9b3ea9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:40.321742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25543c82-464e-4145-b5d3-c8a8f9b3ea9c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:40.329523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 87/181] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:40 2022] GET /identity/v3/role_assignments?user.id=bd58a6d89a6c4fa196454cb9055bbb5c&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:40.402874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20992b-ab73-4070-b768-c77cae29e30e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.403171 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20992b-ab73-4070-b768-c77cae29e30e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.403416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20992b-ab73-4070-b768-c77cae29e30e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.406876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 88/182] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:40.414078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49357d-3347-47e0-8ce1-fee3bd42afa7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.414813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49357d-3347-47e0-8ce1-fee3bd42afa7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.415130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49357d-3347-47e0-8ce1-fee3bd42afa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.438244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 95/183] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:40.452430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3c49357d-3347-47e0-8ce1-fee3bd42afa7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:40.453785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3c49357d-3347-47e0-8ce1-fee3bd42afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:40.492127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af76e0-2e09-42ee-be09-6d08f4b8b30b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:40.557287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 89/184] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:40.558657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117854d-64bf-4d50-8612-1ac8dfc81e8c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.559027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117854d-64bf-4d50-8612-1ac8dfc81e8c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.559514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117854d-64bf-4d50-8612-1ac8dfc81e8c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.560921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 90/185] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:40.569144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb2c5-97e3-48b3-b54b-fe561ff82f04 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.569589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb2c5-97e3-48b3-b54b-fe561ff82f04 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.570027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb2c5-97e3-48b3-b54b-fe561ff82f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.597554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-917cb2c5-97e3-48b3-b54b-fe561ff82f04 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:40.598825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-917cb2c5-97e3-48b3-b54b-fe561ff82f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:40.638368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:40.658402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af76e0-2e09-42ee-be09-6d08f4b8b30b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:40.690318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 91/186] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:40.690959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb2c5-97e3-48b3-b54b-fe561ff82f04 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:40.725548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:40.726188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.726480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.726761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.742105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:40.747410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:40.750025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 96/187] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:00:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:40.750841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:40.763034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:40.816612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb2c5-97e3-48b3-b54b-fe561ff82f04 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:40.860972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:40.861979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.862472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.862932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.865202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e85ecf-e215-40aa-9b2c-8615c47aa770 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:40.874150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:40.879416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:40.880571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:40.882043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:40.884167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 92/188] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:40.885133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:40.901856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:40.902513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:40.902802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:40.903084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:40.904277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:40.905737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:40.907519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 97/189] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:40.908121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:40.920677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:40.995260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:40.996239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531052e-2011-45a8-a28b-465c3b774b9e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.019376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b12033-e18a-4c77-a236-b78163833342 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.034287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.035225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.035659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.036080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.036955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:41.037421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:41.039741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:41.048634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 93/190] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:00:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:41.049673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:41.065468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.066112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.066397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.066676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.067466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:41.067778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:41.069300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:41.078339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 98/191] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:41.079031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:41.156703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:41.166874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7970b9a-4d82-4b3e-b2e2-3fa40eac9fcb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.188191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b1fad-22d8-4fca-9cbe-c3d8fd217c57 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.199688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.200320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.200613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.200894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.201589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:41.203056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:41.222633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a464d5-4689-44af-8060-28db4ba408db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.228026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a464d5-4689-44af-8060-28db4ba408db None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.228026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a464d5-4689-44af-8060-28db4ba408db None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.228026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a464d5-4689-44af-8060-28db4ba408db None admin] PATH_INFO: `/v3/users/7011e4a86677498f8cdeb8e8b159019d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.228026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:41.237536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a464d5-4689-44af-8060-28db4ba408db None admin] RBAC: Authorizing `identity:get_user(user_id=7011e4a86677498f8cdeb8e8b159019d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:41.239282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a464d5-4689-44af-8060-28db4ba408db None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:41.240925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 99/192] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:41 2022] GET /identity/v3/users/7011e4a86677498f8cdeb8e8b159019d => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:41.242221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ee5c2-7d42-4ba5-8141-7c2e91dc6881 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.242512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ee5c2-7d42-4ba5-8141-7c2e91dc6881 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.242792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ee5c2-7d42-4ba5-8141-7c2e91dc6881 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.244179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 100/193] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:41.244795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ee5c2-7d42-4ba5-8141-7c2e91dc6881 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:41.303970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 94/194] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:41 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:41.311536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:41.365557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ee5c2-7d42-4ba5-8141-7c2e91dc6881 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.420987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd5180-0fb5-4de5-ab87-ac119992a49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.421950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5180-0fb5-4de5-ab87-ac119992a49a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.422378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5180-0fb5-4de5-ab87-ac119992a49a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.422811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5180-0fb5-4de5-ab87-ac119992a49a None admin] PATH_INFO: `/v3/users/7011e4a86677498f8cdeb8e8b159019d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.424171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd5180-0fb5-4de5-ab87-ac119992a49a None admin] RBAC: Authorizing `identity:get_user(user_id=7011e4a86677498f8cdeb8e8b159019d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:41.425666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd5180-0fb5-4de5-ab87-ac119992a49a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:41.427440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 101/195] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:41 2022] GET /identity/v3/users/7011e4a86677498f8cdeb8e8b159019d => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:41.427440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e655bf-c42b-4cda-b84e-3a71cb0aec9b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.427440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e655bf-c42b-4cda-b84e-3a71cb0aec9b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.427440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e655bf-c42b-4cda-b84e-3a71cb0aec9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.436593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e45224-60a4-4be8-8fbf-6578ca5520ba None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.449125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b1e655bf-c42b-4cda-b84e-3a71cb0aec9b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:41.450607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b1e655bf-c42b-4cda-b84e-3a71cb0aec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:41.480916 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.481885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.481885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.482097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.504884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:41.539790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:41.551817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 102/196] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:41.553063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e655bf-c42b-4cda-b84e-3a71cb0aec9b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:41.606637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:41.608380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:41.687033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e655bf-c42b-4cda-b84e-3a71cb0aec9b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.736188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.736762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.737298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.737598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.756739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 95/197] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:41.757738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:41.822660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:41.884733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:41.897835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfb079-4e6f-4080-871f-f78ce7df3523 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:41.908570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:41.913039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:41.938336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:41.938990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:41.939320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:41.939603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:41.951772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:41.956868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:41.958620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 96/198] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:41.960381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.087385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5b74b-cf4d-4093-97a6-b19b45f3cc97 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.104209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:42.118838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 103/199] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:42.120249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.139576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.140267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.140554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.140838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.142113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.143637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.145723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 97/200] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:41 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:42.146652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.253204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:42.266006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660afa7-6cba-40ea-8bf6-559f19478e12 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.266882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf58d1-eb46-44a8-9873-700ba368ec7a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.307488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.308293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.308683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.309080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.310531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.312404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.313682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.314487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.314973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.315425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.316284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:42.316725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.318267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.320559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 98/201] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:42.321415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.334476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 104/202] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:42.334476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.382602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:42.468051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fb66a-8e76-492b-a4e0-a4e179079b00 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.476873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c20f7-7f75-4ac9-bc46-84e6b51a6d1f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.513262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.514784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.515473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.516022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.517168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=7011e4a86677498f8cdeb8e8b159019d) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:42.517745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.519503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.519753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.520456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.520816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.521169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.522622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] RBAC: Adding query filter params (role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=7011e4a86677498f8cdeb8e8b159019d) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:42.523075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.524856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.527989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 105/203] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/role_assignments?user.id=7011e4a86677498f8cdeb8e8b159019d&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:42.528992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.542467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 99/204] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/role_assignments?user.id=7011e4a86677498f8cdeb8e8b159019d&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:42.563227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:42.701294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4b88d-9233-4396-8619-e20db17130b8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.760473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.761167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.761455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.761747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] PATH_INFO: `/v3/users/68db2bf12b0b4542846578ee2b7cced5` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:42.762981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] RBAC: Authorizing `identity:get_user(user_id=68db2bf12b0b4542846578ee2b7cced5)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:42.764560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:42.767521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 106/205] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/users/68db2bf12b0b4542846578ee2b7cced5 => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:42.772519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:42.919510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10da266-b6b5-46bf-a9c4-8b3d5a1be5f7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:42.927568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:42.965408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:42.966379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:42.966874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:42.967390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.049671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:43.051611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:43.162377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 100/206] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:43.169838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:43.294860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.316376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dade14-b2ab-489d-81d5-078862c91922 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:43.372982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc67e-c0e4-4e7f-b536-f20df07c98cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:43.373668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc67e-c0e4-4e7f-b536-f20df07c98cb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.373955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc67e-c0e4-4e7f-b536-f20df07c98cb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.374236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc67e-c0e4-4e7f-b536-f20df07c98cb None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.375489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc67e-c0e4-4e7f-b536-f20df07c98cb None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:43.376961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc67e-c0e4-4e7f-b536-f20df07c98cb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:43.378977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 107/207] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:43 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:43.386714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:43.399634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.519739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae74da-560c-4a7b-bb95-7ae4b830f2da None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:43.564905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab4432-2f64-418b-b39e-7f443194fba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:43.565467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4432-2f64-418b-b39e-7f443194fba9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.565749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4432-2f64-418b-b39e-7f443194fba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.566025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4432-2f64-418b-b39e-7f443194fba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.566887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab4432-2f64-418b-b39e-7f443194fba9 None admin] RBAC: Adding query filter params (role.id=0d0f0d7ac63143048ef3044aa9650882, scope.project.id=37f4bbc3aa894e06971fb36f16623f07, user.id=68db2bf12b0b4542846578ee2b7cced5) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:43.567226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab4432-2f64-418b-b39e-7f443194fba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:43.569709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab4432-2f64-418b-b39e-7f443194fba9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:43.577011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 101/208] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:43 2022] GET /identity/v3/role_assignments?user.id=68db2bf12b0b4542846578ee2b7cced5&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=0d0f0d7ac63143048ef3044aa9650882 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:43.695607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b8a0-afe5-4f12-82a9-aa5f66175b5b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.695929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b8a0-afe5-4f12-82a9-aa5f66175b5b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.696174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b8a0-afe5-4f12-82a9-aa5f66175b5b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.697574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 108/209] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:43.705045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb80e6-f131-4dc2-b1b9-19b6270fb0c6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.705335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb80e6-f131-4dc2-b1b9-19b6270fb0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.705615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb80e6-f131-4dc2-b1b9-19b6270fb0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.726584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e2fb80e6-f131-4dc2-b1b9-19b6270fb0c6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:43.727869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e2fb80e6-f131-4dc2-b1b9-19b6270fb0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:43.749818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ce94b-b452-45f6-a908-d81a0225d62e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.750132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ce94b-b452-45f6-a908-d81a0225d62e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.750382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ce94b-b452-45f6-a908-d81a0225d62e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.752127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 109/210] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:43.764420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a27ad-1878-4d0d-bbed-202ff06ddd6b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.764763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a27ad-1878-4d0d-bbed-202ff06ddd6b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.764983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a27ad-1878-4d0d-bbed-202ff06ddd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.794576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c63a27ad-1878-4d0d-bbed-202ff06ddd6b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:43.795825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c63a27ad-1878-4d0d-bbed-202ff06ddd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:43.871326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.876792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 102/211] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:43.906306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb80e6-f131-4dc2-b1b9-19b6270fb0c6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:43.920657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 110/212] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:43.948831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a27ad-1878-4d0d-bbed-202ff06ddd6b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.030682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb80e6-f131-4dc2-b1b9-19b6270fb0c6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.045427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.064860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a27ad-1878-4d0d-bbed-202ff06ddd6b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.071581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.072241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.072524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.072797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.085742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:44.091103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:44.092034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:44.092817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 103/213] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:44.098697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.108777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.109760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.110266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.110756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.112326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.114346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.117340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 111/214] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:44.123265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.172842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.219126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc6b96-c0e2-4447-a071-fdfe5d61cfc2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.242939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a3a80-d200-4d55-9e7a-be20436c2b49 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.263237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.263944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.264228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.264506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.265295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:44.265605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.267030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.281059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 104/215] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:44.286719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.287618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.289080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.289569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.290156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.291158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.292907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.394134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd25b8-58d1-4d26-bf65-b074a7469e9a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.417083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 112/216] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:44 2022] POST /identity/v3/users => generated 297 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:44.423825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.433019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca51e68-703f-4387-bfb4-09f0a4deeecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.433773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca51e68-703f-4387-bfb4-09f0a4deeecf None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.434064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca51e68-703f-4387-bfb4-09f0a4deeecf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.434342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca51e68-703f-4387-bfb4-09f0a4deeecf None admin] PATH_INFO: `/v3/users/356c80633c284cb3982008abf65fe1a5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.451946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca51e68-703f-4387-bfb4-09f0a4deeecf None admin] RBAC: Authorizing `identity:get_user(user_id=356c80633c284cb3982008abf65fe1a5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.453573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca51e68-703f-4387-bfb4-09f0a4deeecf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.459073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 105/217] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/users/356c80633c284cb3982008abf65fe1a5 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:44.463939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.476948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.590500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31966eb5-bf53-46a7-99c7-6c5e02e993f0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.632499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.633657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.634182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.634911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.720532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.722855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.734886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:44.900091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790712da-faaa-488f-9846-780697abea30 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.900393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790712da-faaa-488f-9846-780697abea30 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.900651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790712da-faaa-488f-9846-780697abea30 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.902268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 106/218] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:44.907557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6833a6-5c10-4726-a225-19b30aae818d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.907838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6833a6-5c10-4726-a225-19b30aae818d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.908117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6833a6-5c10-4726-a225-19b30aae818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.911405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 113/219] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:44.918807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.928694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9b6833a6-5c10-4726-a225-19b30aae818d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:44.930110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9b6833a6-5c10-4726-a225-19b30aae818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:45.022516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 107/220] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:45.066809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6833a6-5c10-4726-a225-19b30aae818d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.092122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ad51-da5e-477d-a51a-e62deba38c09 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.109584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:45.150682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.151774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.152262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.152736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.154207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:45.155935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:45.158102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 114/221] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:45.165860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.236600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6833a6-5c10-4726-a225-19b30aae818d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.259597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:45.290900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2e6d4-0551-4245-8528-5b43eb1fb37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.291515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2e6d4-0551-4245-8528-5b43eb1fb37d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.291800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2e6d4-0551-4245-8528-5b43eb1fb37d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.292080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2e6d4-0551-4245-8528-5b43eb1fb37d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.311972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2e6d4-0551-4245-8528-5b43eb1fb37d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9ee2e6d4-0551-4245-8528-5b43eb1fb37d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:45.318569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:45.320769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:45.321790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:45.325373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 108/222] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:00:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:45.327416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b7c1e-b4d5-422c-b3cc-de197bb01c4d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.327740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b7c1e-b4d5-422c-b3cc-de197bb01c4d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.327994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b7c1e-b4d5-422c-b3cc-de197bb01c4d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.329530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 109/223] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:45.335155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b7c1e-b4d5-422c-b3cc-de197bb01c4d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.352872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521e866-e7ff-4ea8-887a-2de7817943d2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.399148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a6d3c-5bc6-4b29-ac3d-6dde18cb2225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.400211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a6d3c-5bc6-4b29-ac3d-6dde18cb2225 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.400706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a6d3c-5bc6-4b29-ac3d-6dde18cb2225 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.401169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a6d3c-5bc6-4b29-ac3d-6dde18cb2225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.402320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a6d3c-5bc6-4b29-ac3d-6dde18cb2225 None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=356c80633c284cb3982008abf65fe1a5) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:45.402840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a6d3c-5bc6-4b29-ac3d-6dde18cb2225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:45.404721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a6d3c-5bc6-4b29-ac3d-6dde18cb2225 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:45.413506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 115/224] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:45 2022] GET /identity/v3/role_assignments?user.id=356c80633c284cb3982008abf65fe1a5&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:45.415109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7452-5192-4ef8-8a69-6775e7afe17d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.415620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7452-5192-4ef8-8a69-6775e7afe17d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.416160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7452-5192-4ef8-8a69-6775e7afe17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.437069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5f4e7452-5192-4ef8-8a69-6775e7afe17d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:45.438701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5f4e7452-5192-4ef8-8a69-6775e7afe17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:45.487802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:45.493013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b7c1e-b4d5-422c-b3cc-de197bb01c4d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.536804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 116/225] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:45.560246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.560836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.561126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.561408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.562650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:45.562973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:45.563612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e7452-5192-4ef8-8a69-6775e7afe17d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.564468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:45.574397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 110/226] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:00:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:45.584290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.679883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:45.688104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e7452-5192-4ef8-8a69-6775e7afe17d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.737573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.738248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.738529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.739062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.764865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:45.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:45.771572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:45.772404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 117/227] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:45.779379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.795484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c246-2a59-4625-ac40-16e0c2a4991a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.832120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.832699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.832988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.833267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] PATH_INFO: `/v3/users/7011e4a86677498f8cdeb8e8b159019d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.834568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] RBAC: Authorizing `identity:get_user(user_id=7011e4a86677498f8cdeb8e8b159019d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:45.836160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:45.837897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 111/228] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:45 2022] GET /identity/v3/users/7011e4a86677498f8cdeb8e8b159019d => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:45.843547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:45.848547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.966059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe33bc-1bb5-460f-859b-cb0d8bdc96d3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.981406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:45.981592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb0b14-75aa-4337-b63b-465e9b4b65a2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:46.015056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.015796 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.016084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.016366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.017623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.019215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.026581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 112/229] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:45 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:46.028200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:46.053048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.053745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.054030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.054311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.055073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:46.055413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.056860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.075727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 118/230] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:46.081651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:46.151715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d817a-d2e9-4987-99d3-31f88bfe2a16 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:46.189646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.191530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.191530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.191530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/7011e4a86677498f8cdeb8e8b159019d/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.193583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] RBAC: Authorizing `identity:create_grant(project_id=5895b755ae0c4e88a8d97321cf0b4411, user_id=7011e4a86677498f8cdeb8e8b159019d, role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.195644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.261986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:46.270753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 113/231] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:46 2022] PUT /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/7011e4a86677498f8cdeb8e8b159019d/roles/9a49213a53964452a411e2dbd74d7c5e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:46.279947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92709ab-d8b5-43e3-860c-b1338f2cfa56 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:46.321179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.321910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.322250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.322537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] PATH_INFO: `/v3/users/7011e4a86677498f8cdeb8e8b159019d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.323836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] RBAC: Authorizing `identity:get_user(user_id=7011e4a86677498f8cdeb8e8b159019d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.325612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.327652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 119/232] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:46 2022] GET /identity/v3/users/7011e4a86677498f8cdeb8e8b159019d => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:46.336439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:46.378126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:46.485394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ee97b-d513-4f3b-a905-6546dbac0091 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:46.535837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ca1c8-ab97-462c-a1ff-f55d67e23e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.536531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ca1c8-ab97-462c-a1ff-f55d67e23e7f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.536823 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ca1c8-ab97-462c-a1ff-f55d67e23e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.537131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ca1c8-ab97-462c-a1ff-f55d67e23e7f None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.538951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ca1c8-ab97-462c-a1ff-f55d67e23e7f None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.540729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ca1c8-ab97-462c-a1ff-f55d67e23e7f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.542928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 114/233] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:46 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:46.551659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:46.681283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d00976-9afe-4c6f-84eb-e643b7639a99 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:46.730284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835df2b2-2298-411d-8001-ef9aeb9ec6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.730984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835df2b2-2298-411d-8001-ef9aeb9ec6fd None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.731299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835df2b2-2298-411d-8001-ef9aeb9ec6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.731584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835df2b2-2298-411d-8001-ef9aeb9ec6fd None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/7011e4a86677498f8cdeb8e8b159019d/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.733642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835df2b2-2298-411d-8001-ef9aeb9ec6fd None admin] RBAC: Authorizing `identity:create_grant(project_id=5895b755ae0c4e88a8d97321cf0b4411, user_id=7011e4a86677498f8cdeb8e8b159019d, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.735201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835df2b2-2298-411d-8001-ef9aeb9ec6fd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.860986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 120/234] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:46 2022] PUT /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/7011e4a86677498f8cdeb8e8b159019d/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:47.066058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170865f-7b93-479b-bd53-e9d44423b64f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.066633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170865f-7b93-479b-bd53-e9d44423b64f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.067307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170865f-7b93-479b-bd53-e9d44423b64f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.071572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 115/235] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:47.078318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c116907-55e3-441e-ac64-b0f74b357a21 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.079037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c116907-55e3-441e-ac64-b0f74b357a21 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.079635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c116907-55e3-441e-ac64-b0f74b357a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.119933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2c116907-55e3-441e-ac64-b0f74b357a21 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:47.121707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2c116907-55e3-441e-ac64-b0f74b357a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:47.150991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.204662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee415e-e75d-42a2-8378-357e3f734f1f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.205193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee415e-e75d-42a2-8378-357e3f734f1f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.205643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee415e-e75d-42a2-8378-357e3f734f1f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.208467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 116/236] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:47.220194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735da6-5772-4c70-8fac-0cb763995218 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.220692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735da6-5772-4c70-8fac-0cb763995218 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.221171 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735da6-5772-4c70-8fac-0cb763995218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.247843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 121/237] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:47.253055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fa735da6-5772-4c70-8fac-0cb763995218 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:47.254835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fa735da6-5772-4c70-8fac-0cb763995218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:47.270968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c116907-55e3-441e-ac64-b0f74b357a21 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.379507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c116907-55e3-441e-ac64-b0f74b357a21 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.404130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 117/238] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:47.429001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.429685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.429976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.430262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.453243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:47.458463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:47.461145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:47.463076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 122/239] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:47.467794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735da6-5772-4c70-8fac-0cb763995218 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.472217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.493013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.502231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.663802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735da6-5772-4c70-8fac-0cb763995218 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.666450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a40f9-06af-4660-a145-5746a07a4632 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.728499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.729160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.729443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.729718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.730866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:47.732367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:47.734161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 123/240] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:47.739694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.756148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.757334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.757619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.757891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.762526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:47.762855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:47.764298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:47.783961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 118/241] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:47.789875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.882062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.923260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa590c0-487d-4b3d-9e22-ae7fa28daf90 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.971903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.994285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255a741-1517-4bd9-b18f-1134d681a6af None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.007514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6de1b9-37c7-4563-a512-11e574c4e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.008191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6de1b9-37c7-4563-a512-11e574c4e69b None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.010575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6de1b9-37c7-4563-a512-11e574c4e69b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.011087 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6de1b9-37c7-4563-a512-11e574c4e69b None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.013345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6de1b9-37c7-4563-a512-11e574c4e69b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.014822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6de1b9-37c7-4563-a512-11e574c4e69b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.038788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e0cc8-5979-42dd-93fc-48e72bcb1ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.039792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e0cc8-5979-42dd-93fc-48e72bcb1ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.043783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e0cc8-5979-42dd-93fc-48e72bcb1ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.043783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e0cc8-5979-42dd-93fc-48e72bcb1ec7 None admin] PATH_INFO: `/v3/users/c3ded8f94dbd44a3aab7c0a55be919d2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.059285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e0cc8-5979-42dd-93fc-48e72bcb1ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=c3ded8f94dbd44a3aab7c0a55be919d2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.061060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e0cc8-5979-42dd-93fc-48e72bcb1ec7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.063349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 119/242] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/users/c3ded8f94dbd44a3aab7c0a55be919d2 => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:48.064918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e971ba-c3c7-4f76-8c0b-88fe80ff1014 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.065466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e971ba-c3c7-4f76-8c0b-88fe80ff1014 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.065963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e971ba-c3c7-4f76-8c0b-88fe80ff1014 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.067400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 120/243] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:48.072006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e971ba-c3c7-4f76-8c0b-88fe80ff1014 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.125367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 124/244] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/users => generated 293 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:48.126536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ff203-3ba2-415f-8983-a3e24c7d0508 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.126838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ff203-3ba2-415f-8983-a3e24c7d0508 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.127123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ff203-3ba2-415f-8983-a3e24c7d0508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.149115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3e3ff203-3ba2-415f-8983-a3e24c7d0508 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:48.150386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3e3ff203-3ba2-415f-8983-a3e24c7d0508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:48.206152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e971ba-c3c7-4f76-8c0b-88fe80ff1014 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.239123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 125/245] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:48.252133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.253087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.253613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.254087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.263207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ff203-3ba2-415f-8983-a3e24c7d0508 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.276215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.376863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.392146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.394630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.432277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ff203-3ba2-415f-8983-a3e24c7d0508 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.476165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.477205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.477694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.478180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.490586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:48.496902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:48.497902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:48.498699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 126/246] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:48.505064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.530053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 121/247] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:48.536923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.606788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.667122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822f93-5039-4463-99a6-83c89d8205aa None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.685330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ecad9-da9e-4cde-94f7-60cdc967d279 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.695971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.711112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.711791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.712084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.712371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.713191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:48.713506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.715823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.728609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 127/248] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:48.736251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.742430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.743321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.743632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.747370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.747370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.747370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.748683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 122/249] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:48.754643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.872850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0366f4-013d-4134-b779-4f9951f0ad3b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.880608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6267e19-e454-4274-aed1-f17261b8fa7c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.902402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.922981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.923691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.923971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.924324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] PATH_INFO: `/v3/users/356c80633c284cb3982008abf65fe1a5` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.925582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] RBAC: Authorizing `identity:get_user(user_id=356c80633c284cb3982008abf65fe1a5)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.927058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.929016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 128/250] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/users/356c80633c284cb3982008abf65fe1a5 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:48.929798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.935310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.935940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.936249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.936639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.938888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] RBAC: Adding query filter params (role.id=0d0f0d7ac63143048ef3044aa9650882, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=c3ded8f94dbd44a3aab7c0a55be919d2) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:48.939479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.941519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.949056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 123/251] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/role_assignments?user.id=c3ded8f94dbd44a3aab7c0a55be919d2&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=0d0f0d7ac63143048ef3044aa9650882 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:49.011802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:49.082035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20134f3b-af04-41ad-92bf-6d7812db2108 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.140004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6f51e-f83b-45c8-8114-1dd00a3b72e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.140698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6f51e-f83b-45c8-8114-1dd00a3b72e4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.140985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6f51e-f83b-45c8-8114-1dd00a3b72e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.141273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6f51e-f83b-45c8-8114-1dd00a3b72e4 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.142655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6f51e-f83b-45c8-8114-1dd00a3b72e4 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.144176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6f51e-f83b-45c8-8114-1dd00a3b72e4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.146243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 129/252] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:49.152194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.167243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccdca0-7b22-44f0-84fb-baebbafe1404 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.167649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccdca0-7b22-44f0-84fb-baebbafe1404 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.167825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccdca0-7b22-44f0-84fb-baebbafe1404 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.169987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 130/253] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:49.183886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c841214-1c15-4ced-a2ef-ff592517ba24 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.184204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c841214-1c15-4ced-a2ef-ff592517ba24 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.184461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c841214-1c15-4ced-a2ef-ff592517ba24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.207063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5c841214-1c15-4ced-a2ef-ff592517ba24 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:49.208756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5c841214-1c15-4ced-a2ef-ff592517ba24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.318365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 131/254] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:49.388705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c841214-1c15-4ced-a2ef-ff592517ba24 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.389946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3e895-1f18-4616-96fd-f95aa3afb904 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.468946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d18d9-2fc0-4f17-82c7-2fdca88b6f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.469664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d18d9-2fc0-4f17-82c7-2fdca88b6f5d None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.469957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d18d9-2fc0-4f17-82c7-2fdca88b6f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.470240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d18d9-2fc0-4f17-82c7-2fdca88b6f5d None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/356c80633c284cb3982008abf65fe1a5/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.476098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d18d9-2fc0-4f17-82c7-2fdca88b6f5d None admin] RBAC: Authorizing `identity:create_grant(project_id=5895b755ae0c4e88a8d97321cf0b4411, user_id=356c80633c284cb3982008abf65fe1a5, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.477612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d18d9-2fc0-4f17-82c7-2fdca88b6f5d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.491284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:49.551954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 124/255] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:49 2022] PUT /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/356c80633c284cb3982008abf65fe1a5/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:49.553378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb68df6-f5a1-448b-af0a-7ee7f068093b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.553685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb68df6-f5a1-448b-af0a-7ee7f068093b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.554017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb68df6-f5a1-448b-af0a-7ee7f068093b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.555431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 125/256] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:49.562626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49110b9-f64c-44ff-a727-1764e3a63dba None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.562929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49110b9-f64c-44ff-a727-1764e3a63dba None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.563216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49110b9-f64c-44ff-a727-1764e3a63dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.583430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c841214-1c15-4ced-a2ef-ff592517ba24 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.601972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a49110b9-f64c-44ff-a727-1764e3a63dba None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:49.603212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a49110b9-f64c-44ff-a727-1764e3a63dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.633743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.634435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.634722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.635003 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.656335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:49.661806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:49.662731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:00:49.664024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:49.671906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 132/257] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:00:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:49.672902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.702422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:49.789571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 126/258] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:49.823934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49110b9-f64c-44ff-a727-1764e3a63dba None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.870895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1c8a-de7e-494f-9da9-56f9a0c831e8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.895344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:49.937162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.937815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.938162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.938444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.939255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:49.939573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.940997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.959147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 133/259] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:00:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:49.964748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.046832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49110b9-f64c-44ff-a727-1764e3a63dba None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.130989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3719e-12d4-4270-a553-82c9a59b9dca None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.139919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.140592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.140875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.141149 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.168345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:50.175347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:50.176456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:50.177589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 127/260] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:50.185776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.200621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.201287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.201568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.201841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] PATH_INFO: `/v3/users/7011e4a86677498f8cdeb8e8b159019d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.203058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] RBAC: Authorizing `identity:get_user(user_id=7011e4a86677498f8cdeb8e8b159019d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.204535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.206518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 134/261] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:49 2022] GET /identity/v3/users/7011e4a86677498f8cdeb8e8b159019d => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:50.216246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.268185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.311556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.419776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52109eb-a50a-45c8-a3f9-e0f3d44d24fc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.460926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07de2b-ed59-48c3-8efc-12cbb114081c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.489126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.489771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.490057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.490339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.491121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:50.491456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.492871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.511387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 128/262] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:50.515883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.535794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.536462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.536748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.537033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.676305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.677905 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.679538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.703655 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0661a1d-4f17-4e78-8514-7db17862848c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.756348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.807961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.808642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.808938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.809227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] PATH_INFO: `/v3/users/7011e4a86677498f8cdeb8e8b159019d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.810495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] RBAC: Authorizing `identity:get_user(user_id=7011e4a86677498f8cdeb8e8b159019d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.811992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.814788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 129/263] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:50 2022] GET /identity/v3/users/7011e4a86677498f8cdeb8e8b159019d => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:50.820524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.875838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 135/264] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:50.882355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.056880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4681f5-5397-430e-bc9a-53709fadc262 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.060122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd3c59-15ab-4e43-9ca8-2c0d183b1933 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.125175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.125826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.126111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.126389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.128161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.128811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.129087 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.129358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.132736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.134253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.138191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 136/265] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:50 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:51.144194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.167584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.173896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.223663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.225822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.348294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 130/266] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:00:51.356413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.366886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99a223-5b2b-40dd-9854-b34350a1da7b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.429395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.430034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.430314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.430582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.431408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] RBAC: Adding query filter params (role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=7011e4a86677498f8cdeb8e8b159019d) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:51.431718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.433402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.445168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 137/267] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/role_assignments?user.id=7011e4a86677498f8cdeb8e8b159019d&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:51.468862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.527060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be21065-b863-4de2-a25a-bc62ad4e59ab None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.577600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b055cc-e779-4c07-9d0c-59289ac6faee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.578274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b055cc-e779-4c07-9d0c-59289ac6faee None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.578619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b055cc-e779-4c07-9d0c-59289ac6faee None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.578900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b055cc-e779-4c07-9d0c-59289ac6faee None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.580176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b055cc-e779-4c07-9d0c-59289ac6faee None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.581677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b055cc-e779-4c07-9d0c-59289ac6faee None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.583461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 131/268] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:51.590393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.625248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf0db3-4994-4f74-991c-6e357ff202b3 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.625872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf0db3-4994-4f74-991c-6e357ff202b3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.626350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf0db3-4994-4f74-991c-6e357ff202b3 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.627987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 132/269] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:51.633767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687e6f4-9ae0-4ca3-b884-df8c90f874fa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.634382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687e6f4-9ae0-4ca3-b884-df8c90f874fa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.634876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687e6f4-9ae0-4ca3-b884-df8c90f874fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.655056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7687e6f4-9ae0-4ca3-b884-df8c90f874fa None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:51.656603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7687e6f4-9ae0-4ca3-b884-df8c90f874fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.707628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.733961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 133/270] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:51.741932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd8264-ec3d-4fc7-bc54-a5149f8baf0a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.758052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687e6f4-9ae0-4ca3-b884-df8c90f874fa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.781857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.782811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.783287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.783727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.784675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=7011e4a86677498f8cdeb8e8b159019d) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:51.785138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.786742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.794653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 138/271] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/role_assignments?user.id=7011e4a86677498f8cdeb8e8b159019d&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:51.829150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.883809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687e6f4-9ae0-4ca3-b884-df8c90f874fa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.925041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918db648-d14f-4088-9699-3d8ae50ea408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.926143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db648-d14f-4088-9699-3d8ae50ea408 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.926429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db648-d14f-4088-9699-3d8ae50ea408 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.926708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db648-d14f-4088-9699-3d8ae50ea408 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.928032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918db648-d14f-4088-9699-3d8ae50ea408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.930123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918db648-d14f-4088-9699-3d8ae50ea408 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.932236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 134/272] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:51.937898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.060712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b527c-d057-4f96-9c1d-2120e58e16e2 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.061033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b527c-d057-4f96-9c1d-2120e58e16e2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.061319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b527c-d057-4f96-9c1d-2120e58e16e2 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.064106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 135/273] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:52.066268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aa3dc-f4da-464f-b79b-d013fc919abf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.077135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a689d-563f-47f8-9bb3-3117f8073842 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.077464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a689d-563f-47f8-9bb3-3117f8073842 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.077703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a689d-563f-47f8-9bb3-3117f8073842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.112718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d05a689d-563f-47f8-9bb3-3117f8073842 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:52.114397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d05a689d-563f-47f8-9bb3-3117f8073842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:52.119445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.120100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.120390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.120671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.121457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.122964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.143109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:52.197255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 136/274] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:52.198418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58933-2794-400b-92ad-f1b81e7ece7e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.198710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58933-2794-400b-92ad-f1b81e7ece7e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.198988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58933-2794-400b-92ad-f1b81e7ece7e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.200520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 137/275] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:52.206039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fabe9b-8453-4906-8066-727b0d99a0a3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.206325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fabe9b-8453-4906-8066-727b0d99a0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.206599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fabe9b-8453-4906-8066-727b0d99a0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.228447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f3fabe9b-8453-4906-8066-727b0d99a0a3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:52.229654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f3fabe9b-8453-4906-8066-727b0d99a0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:52.289465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:52.309239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 138/276] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:52.310109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fabe9b-8453-4906-8066-727b0d99a0a3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.330972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 139/277] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:00:51 2022] POST /identity/v3/users => generated 293 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:52.342197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.454057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fabe9b-8453-4906-8066-727b0d99a0a3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.492474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d77be-0ffb-4e86-bbaa-932dfa0fbdb2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.504044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.504044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.504339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.504567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.521180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:52.528115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:52.529115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:52.529886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 139/278] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:52.539533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.551573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.552270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.552522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.552801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.568453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:52.574677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:52.575670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:52.576474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 140/279] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:52.581801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.599874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:52.661865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:52.684277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725db63-c32b-4049-b587-5217f082bd44 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.729365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540238-f0e0-4749-b71b-7b421429381e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.740378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a8efc-db4f-4d78-8d3f-8ec15efb4e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.741147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8efc-db4f-4d78-8d3f-8ec15efb4e49 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.741642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8efc-db4f-4d78-8d3f-8ec15efb4e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.741746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8efc-db4f-4d78-8d3f-8ec15efb4e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.742423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a8efc-db4f-4d78-8d3f-8ec15efb4e49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:52.742728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a8efc-db4f-4d78-8d3f-8ec15efb4e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.744234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a8efc-db4f-4d78-8d3f-8ec15efb4e49 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.756156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 140/280] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:52.758121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e96e-6838-4d35-9b89-c076964b4123 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.758414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e96e-6838-4d35-9b89-c076964b4123 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.758771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e96e-6838-4d35-9b89-c076964b4123 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.764570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 141/281] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:52.765350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526e96e-6838-4d35-9b89-c076964b4123 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.775641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6ba1-5b5c-438a-81ed-58fe033f0363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.776285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6ba1-5b5c-438a-81ed-58fe033f0363 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.776575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6ba1-5b5c-438a-81ed-58fe033f0363 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.776853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6ba1-5b5c-438a-81ed-58fe033f0363 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.777583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6ba1-5b5c-438a-81ed-58fe033f0363 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:52.777896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6ba1-5b5c-438a-81ed-58fe033f0363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.779823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6ba1-5b5c-438a-81ed-58fe033f0363 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.793164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 141/282] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:52.794698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50fc23-6dce-4463-9e9e-be0bf6b8331e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.795230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50fc23-6dce-4463-9e9e-be0bf6b8331e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.795754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50fc23-6dce-4463-9e9e-be0bf6b8331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.816224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3d50fc23-6dce-4463-9e9e-be0bf6b8331e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:52.817864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3d50fc23-6dce-4463-9e9e-be0bf6b8331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:52.892632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526e96e-6838-4d35-9b89-c076964b4123 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.897251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 142/283] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:52.898123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50fc23-6dce-4463-9e9e-be0bf6b8331e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.901307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:52.933403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.934478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.935089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.935675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] PATH_INFO: `/v3/users/c3ded8f94dbd44a3aab7c0a55be919d2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.937082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] RBAC: Authorizing `identity:get_user(user_id=c3ded8f94dbd44a3aab7c0a55be919d2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.938872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.941170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 142/284] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/users/c3ded8f94dbd44a3aab7c0a55be919d2 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:52.942157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.961300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:53.038030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50fc23-6dce-4463-9e9e-be0bf6b8331e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.060354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee17b0-7eb9-43d5-9214-f57b31c20753 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.083755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.084425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.084743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.085031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] PATH_INFO: `/v3/users/d88064b1f2294b30b02957189db19554` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.102735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.103374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] RBAC: Authorizing `identity:get_user(user_id=d88064b1f2294b30b02957189db19554)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.103481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.103689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.103970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.105525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.107627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 143/285] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/users/d88064b1f2294b30b02957189db19554 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:53.108641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:53.117932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:53.124514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:53.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:53.126224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 143/286] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:53.126868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:53.177063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:53.222854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31f21-dae8-45e1-b448-842bc2f64e87 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.225657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:53.257565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c005bc4-331c-45df-a3d8-bdf602cd6d66 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.262165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.263059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.263561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.264082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.265492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.267163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.269150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 144/287] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:53 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:53.270166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:53.299199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.299757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.300042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.300364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.301099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, user.id=d88064b1f2294b30b02957189db19554, scope.system=all) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:53.301410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.302863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.308958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 144/288] 10.222.0.49 () {56 vars in 1308 bytes} [Tue Aug 2 10:00:53 2022] GET /identity/v3/role_assignments?user.id=d88064b1f2294b30b02957189db19554&scope.system=all&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:53.309769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:53.389182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13510605-8c20-439d-b32c-e72dc0e8f6a4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.414684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:53.418512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cccbf-58a9-4ebc-bb6d-f720aedfed77 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.431353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.432376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.432859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.433338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.434292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:53.434902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.436667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.446946 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 145/289] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:53.452230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:53.458196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.458893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.459181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.459493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/c3ded8f94dbd44a3aab7c0a55be919d2/roles/0d0f0d7ac63143048ef3044aa9650882` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.461573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5895b755ae0c4e88a8d97321cf0b4411, user_id=c3ded8f94dbd44a3aab7c0a55be919d2, role_id=0d0f0d7ac63143048ef3044aa9650882)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.463052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.495154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:53.547903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 145/290] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:00:53 2022] PUT /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/users/c3ded8f94dbd44a3aab7c0a55be919d2/roles/0d0f0d7ac63143048ef3044aa9650882 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:53.571235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363f49b-854b-4f6d-a437-66989c14088b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.611492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.612362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.612798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.613080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] PATH_INFO: `/v3/users/356c80633c284cb3982008abf65fe1a5` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.614635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] RBAC: Authorizing `identity:get_user(user_id=356c80633c284cb3982008abf65fe1a5)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.616210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.618466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 146/291] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:53 2022] GET /identity/v3/users/356c80633c284cb3982008abf65fe1a5 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:53.623963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:53.767985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11d401-1441-4d6e-9ddc-769f6b657de9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.810651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.811336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.811627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.812012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.842923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:53.887017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.889422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.994804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 146/292] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:54.000228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:54.143736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5e4ba-5623-41d4-89c4-92780d01a5d1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:54.145245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:54.196535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b4b59-0e1a-4739-bd38-df3b0827c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:54.197208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b4b59-0e1a-4739-bd38-df3b0827c9ea None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:54.197500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b4b59-0e1a-4739-bd38-df3b0827c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:54.197786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b4b59-0e1a-4739-bd38-df3b0827c9ea None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:54.198983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b4b59-0e1a-4739-bd38-df3b0827c9ea None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:54.200482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b4b59-0e1a-4739-bd38-df3b0827c9ea None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:54.202519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 147/293] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:53 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:54.208491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:54.326165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:54.408172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46098221-e28b-4dbe-83dc-a1c041244ffa None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:54.480943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c7e1f-1509-46c2-aeb4-ce73619b8d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:54.481573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c7e1f-1509-46c2-aeb4-ce73619b8d61 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:54.481855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c7e1f-1509-46c2-aeb4-ce73619b8d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:54.482129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c7e1f-1509-46c2-aeb4-ce73619b8d61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:54.482927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c7e1f-1509-46c2-aeb4-ce73619b8d61 None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=356c80633c284cb3982008abf65fe1a5) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:54.483255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c7e1f-1509-46c2-aeb4-ce73619b8d61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:54.484703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c7e1f-1509-46c2-aeb4-ce73619b8d61 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:54.496148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 147/294] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:54 2022] GET /identity/v3/role_assignments?user.id=356c80633c284cb3982008abf65fe1a5&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:54.802197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551f44-20b9-4e2d-9fc0-51fb430af8e3 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:54.802576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551f44-20b9-4e2d-9fc0-51fb430af8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:54.802759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551f44-20b9-4e2d-9fc0-51fb430af8e3 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:54.804981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 148/295] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:54.810542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd25b47-2909-4844-897c-b680edc7f0e2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:54.811453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd25b47-2909-4844-897c-b680edc7f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:54.811957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd25b47-2909-4844-897c-b680edc7f0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:54.832110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2cd25b47-2909-4844-897c-b680edc7f0e2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:54.833632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2cd25b47-2909-4844-897c-b680edc7f0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:54.839948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce856a4e-45d4-4208-b50a-23ece61f2b8d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:54.840456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce856a4e-45d4-4208-b50a-23ece61f2b8d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:54.840908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce856a4e-45d4-4208-b50a-23ece61f2b8d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:54.842733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 149/296] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:54.848455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a174e2-e2ac-4ab4-9ef5-d35db5ff69ac None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:54.848993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a174e2-e2ac-4ab4-9ef5-d35db5ff69ac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:54.849467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a174e2-e2ac-4ab4-9ef5-d35db5ff69ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:54.869141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d1a174e2-e2ac-4ab4-9ef5-d35db5ff69ac None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:54.870847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d1a174e2-e2ac-4ab4-9ef5-d35db5ff69ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:54.921655 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 148/297] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:54.946605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd25b47-2909-4844-897c-b680edc7f0e2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:54.951428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 150/298] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:54.977931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a174e2-e2ac-4ab4-9ef5-d35db5ff69ac None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.022023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:55.094079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd25b47-2909-4844-897c-b680edc7f0e2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.130160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:55.134442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a174e2-e2ac-4ab4-9ef5-d35db5ff69ac None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.134878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.137510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.138012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.138490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.140002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:55.141680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:55.143837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 149/299] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:55.150614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.175590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.176257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.176540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.176817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.187909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:55.192940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:55.193896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:55.195306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:55.199346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 151/300] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:55.200813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.311564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b93d-3917-4a17-a975-d1b830d12c08 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.316110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6f727-2704-4164-9c79-89b7c7066e67 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.337956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:55.357263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.357263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.357263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.357263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] PATH_INFO: `/v3/groups` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.357668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:55.357923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:55.365414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.366993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.367354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.367656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.368453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:55.368794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:55.370315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:55.379949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 152/301] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:55.386291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.412132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:55.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 150/302] 10.222.0.49 () {60 vars in 1134 bytes} [Tue Aug 2 10:00:55 2022] POST /identity/v3/groups => generated 229 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:55.516092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d2c36-d283-4154-978f-ae559f0540b7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.557096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73255f8e-5c64-4a2d-8f32-82109b46f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.557817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73255f8e-5c64-4a2d-8f32-82109b46f12b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.558104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73255f8e-5c64-4a2d-8f32-82109b46f12b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.558385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73255f8e-5c64-4a2d-8f32-82109b46f12b None admin] PATH_INFO: `/v3/users/b756d3dfe0ee4c10bebd90bc370b6c50` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.574578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73255f8e-5c64-4a2d-8f32-82109b46f12b None admin] RBAC: Authorizing `identity:get_user(user_id=b756d3dfe0ee4c10bebd90bc370b6c50)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:55.579302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73255f8e-5c64-4a2d-8f32-82109b46f12b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:55.579302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 153/303] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:55 2022] GET /identity/v3/users/b756d3dfe0ee4c10bebd90bc370b6c50 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:55.587213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.609015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87faa6b-2542-4c51-baee-ee190896ade4 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.609015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87faa6b-2542-4c51-baee-ee190896ade4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.609355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87faa6b-2542-4c51-baee-ee190896ade4 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.610792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 154/304] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:55.616351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a45ce-9a12-4f0d-ad8f-1b296a2fe734 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.616639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a45ce-9a12-4f0d-ad8f-1b296a2fe734 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.617654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a45ce-9a12-4f0d-ad8f-1b296a2fe734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.635818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-116a45ce-9a12-4f0d-ad8f-1b296a2fe734 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:55.637027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-116a45ce-9a12-4f0d-ad8f-1b296a2fe734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:55.714786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4c8f5-7cd4-433b-83e6-63f69017cc99 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.729292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 155/305] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:55.752044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a45ce-9a12-4f0d-ad8f-1b296a2fe734 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:55.757924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.757924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.757924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.757924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.758243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] RBAC: Adding query filter params (role.id=0d0f0d7ac63143048ef3044aa9650882, user.id=b756d3dfe0ee4c10bebd90bc370b6c50, scope.system=all) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:55.758243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:55.759410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:55.765546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:55.768169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 151/306] 10.222.0.49 () {56 vars in 1308 bytes} [Tue Aug 2 10:00:55 2022] GET /identity/v3/role_assignments?user.id=b756d3dfe0ee4c10bebd90bc370b6c50&scope.system=all&role.id=0d0f0d7ac63143048ef3044aa9650882 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:55.875211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a45ce-9a12-4f0d-ad8f-1b296a2fe734 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:55.914398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:55.915072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:55.915391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:55.915672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:55.928589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:55.934012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:55.934899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:55.936154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 156/307] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:55.941489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.000848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.058879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b06e8d-aad5-4c8f-b774-a6a6695ca100 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.103079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc59f3-604e-4515-a0ae-2b476dfdebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.104115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc59f3-604e-4515-a0ae-2b476dfdebf9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.104614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc59f3-604e-4515-a0ae-2b476dfdebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.105088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc59f3-604e-4515-a0ae-2b476dfdebf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.106025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc59f3-604e-4515-a0ae-2b476dfdebf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:56.106544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc59f3-604e-4515-a0ae-2b476dfdebf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.108168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc59f3-604e-4515-a0ae-2b476dfdebf9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.118537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 152/308] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.124073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.196282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.229051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52844ac4-8902-4e42-ab9c-9616619eae2e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.260805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909a4d8-6e4f-4c3a-8f89-4b8ad80c6da3 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.261090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909a4d8-6e4f-4c3a-8f89-4b8ad80c6da3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.261532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909a4d8-6e4f-4c3a-8f89-4b8ad80c6da3 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.262896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 153/309] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:56.268180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50031bbf-4983-4172-a827-6489d1f3cddd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.268734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50031bbf-4983-4172-a827-6489d1f3cddd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.269029 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.269436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50031bbf-4983-4172-a827-6489d1f3cddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.269692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.269982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.270267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] PATH_INFO: `/v3/users/d88064b1f2294b30b02957189db19554` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.271557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] RBAC: Authorizing `identity:get_user(user_id=d88064b1f2294b30b02957189db19554)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.273602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.275804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 157/310] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/users/d88064b1f2294b30b02957189db19554 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:56.281470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.290538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-50031bbf-4983-4172-a827-6489d1f3cddd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:56.292052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-50031bbf-4983-4172-a827-6489d1f3cddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:56.381410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 154/311] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:56.393813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300fe06-ec98-484c-8198-86362415fb66 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.404717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50031bbf-4983-4172-a827-6489d1f3cddd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.434533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.435725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.436513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.437250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] PATH_INFO: `/v3/system/users/d88064b1f2294b30b02957189db19554/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.439361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d88064b1f2294b30b02957189db19554, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.441206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.443554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.513970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50031bbf-4983-4172-a827-6489d1f3cddd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.517968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 158/312] 10.222.0.49 () {58 vars in 1334 bytes} [Tue Aug 2 10:00:56 2022] PUT /identity/v3/system/users/d88064b1f2294b30b02957189db19554/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:56.561551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.562268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.562556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.562837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.576235 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:56.581728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:56.582671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:56.583499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 155/313] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:56.590177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.593305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.708547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b26f5-03b6-4fc2-a2ed-575ea9a6681b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.745645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.746344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.746631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.746907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.747696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:56.748032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.749461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.760165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 159/314] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.767044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.873248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:56.879997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25f63b-84e9-414b-8dc8-6986127ca649 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:56.915791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:56.916487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.916774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.917046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] PATH_INFO: `/v3/users/c3ded8f94dbd44a3aab7c0a55be919d2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.918226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] RBAC: Authorizing `identity:get_user(user_id=c3ded8f94dbd44a3aab7c0a55be919d2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:56.919731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:56.921622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 156/315] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/users/c3ded8f94dbd44a3aab7c0a55be919d2 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:56.927346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:56.996635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.045455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60dbb-927e-4a48-8d42-534bb878015d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.080211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.081124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.081577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.082018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.157780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.159381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.224488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.270475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 160/316] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:00:57.271801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.384954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e47355-0b2a-4c18-b9c2-6f30eecdd2a3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.422806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ffdb0-65a1-4a13-8fc1-dddb64590e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.423485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ffdb0-65a1-4a13-8fc1-dddb64590e57 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.423772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ffdb0-65a1-4a13-8fc1-dddb64590e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.424053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ffdb0-65a1-4a13-8fc1-dddb64590e57 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.425215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ffdb0-65a1-4a13-8fc1-dddb64590e57 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.426670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ffdb0-65a1-4a13-8fc1-dddb64590e57 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.428367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 157/317] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:57.434541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.556969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455d020-60a5-45a6-80f8-60f616a0300a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.594567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49259b01-ad99-4a7c-b253-f4d7267237e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.595602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49259b01-ad99-4a7c-b253-f4d7267237e5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.596126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49259b01-ad99-4a7c-b253-f4d7267237e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.596621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49259b01-ad99-4a7c-b253-f4d7267237e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.597684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49259b01-ad99-4a7c-b253-f4d7267237e5 None admin] RBAC: Adding query filter params (role.id=0d0f0d7ac63143048ef3044aa9650882, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411, user.id=c3ded8f94dbd44a3aab7c0a55be919d2) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:57.598232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49259b01-ad99-4a7c-b253-f4d7267237e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.599958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49259b01-ad99-4a7c-b253-f4d7267237e5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.606913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 161/318] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/role_assignments?user.id=c3ded8f94dbd44a3aab7c0a55be919d2&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=0d0f0d7ac63143048ef3044aa9650882 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:57.657746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67202-3d08-494c-ad95-61d5193d5dca None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.658396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67202-3d08-494c-ad95-61d5193d5dca None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.658883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67202-3d08-494c-ad95-61d5193d5dca None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.660904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 158/319] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:57.667009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a44ee-3d50-4407-806c-8bb974dac808 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.667352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a44ee-3d50-4407-806c-8bb974dac808 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.667657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a44ee-3d50-4407-806c-8bb974dac808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.688797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-624a44ee-3d50-4407-806c-8bb974dac808 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:57.690429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-624a44ee-3d50-4407-806c-8bb974dac808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:57.719037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.774794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 162/320] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:57.799444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a67202-3d08-494c-ad95-61d5193d5dca None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:57.839136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:57.900519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a355-efb2-4d23-b04e-fb893b96c444 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.900811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a355-efb2-4d23-b04e-fb893b96c444 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.901090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a355-efb2-4d23-b04e-fb893b96c444 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.902461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 163/321] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:00:57.908103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915cb82-2538-450a-aaa2-3b0a7db8eb54 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.908552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915cb82-2538-450a-aaa2-3b0a7db8eb54 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.908975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915cb82-2538-450a-aaa2-3b0a7db8eb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.924337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a67202-3d08-494c-ad95-61d5193d5dca None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:57.928180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b915cb82-2538-450a-aaa2-3b0a7db8eb54 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:57.930102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b915cb82-2538-450a-aaa2-3b0a7db8eb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:57.959762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:57.960414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:57.960701 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:57.960988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:57.962124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:57.963636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:57.965237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 159/322] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:57.970829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.001130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 164/323] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:00:58.023606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915cb82-2538-450a-aaa2-3b0a7db8eb54 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.080883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.084850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fa72a-107b-4d65-aabb-daa90bff401a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.111687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.123531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.124167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.124358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915cb82-2538-450a-aaa2-3b0a7db8eb54 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.124455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.124718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] PATH_INFO: `/v3/groups` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.125429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.126886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.166574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.167197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.167506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.167811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.179984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:58.185724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:58.191823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:00:58.192639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 165/324] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:00:58.194704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.251855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 160/325] 10.222.0.49 () {60 vars in 1134 bytes} [Tue Aug 2 10:00:57 2022] POST /identity/v3/groups => generated 226 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:58.320094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11dc8-63ed-48da-9e43-4ed1ae736999 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.360409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.361435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.361916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.362386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.363398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:58.363914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.365845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.376489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 166/326] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:58.382755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.544065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febc4da-2b4f-45f2-9536-4bf5926fdc44 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.588791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce6ddf-ffcd-494d-bfa8-3306ff7cc766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.589742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce6ddf-ffcd-494d-bfa8-3306ff7cc766 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.590032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce6ddf-ffcd-494d-bfa8-3306ff7cc766 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.590313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce6ddf-ffcd-494d-bfa8-3306ff7cc766 None admin] PATH_INFO: `/v3/users/b756d3dfe0ee4c10bebd90bc370b6c50` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.591541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce6ddf-ffcd-494d-bfa8-3306ff7cc766 None admin] RBAC: Authorizing `identity:get_user(user_id=b756d3dfe0ee4c10bebd90bc370b6c50)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.593004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce6ddf-ffcd-494d-bfa8-3306ff7cc766 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.594773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 161/327] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/users/b756d3dfe0ee4c10bebd90bc370b6c50 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:58.600276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.613581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.695254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc60d7-11f3-4184-9220-992c5856871a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.695955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc60d7-11f3-4184-9220-992c5856871a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.696371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc60d7-11f3-4184-9220-992c5856871a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.698299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 162/328] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:58.707368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a834e-790a-463e-b99c-5424a93a6b92 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.708179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a834e-790a-463e-b99c-5424a93a6b92 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.708869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a834e-790a-463e-b99c-5424a93a6b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.724360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98ad2f-b736-4981-a105-ec24f5a5fd38 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:58.738253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-565a834e-790a-463e-b99c-5424a93a6b92 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:58.740674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-565a834e-790a-463e-b99c-5424a93a6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:58.767158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:58.767916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:58.768198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:58.768483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] PATH_INFO: `/v3/system/users/b756d3dfe0ee4c10bebd90bc370b6c50/roles/0d0f0d7ac63143048ef3044aa9650882` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:58.769983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b756d3dfe0ee4c10bebd90bc370b6c50, role_id=0d0f0d7ac63143048ef3044aa9650882)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:58.771478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:58.772881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:58.826709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 163/329] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:00:58.826890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 167/330] 10.222.0.49 () {58 vars in 1334 bytes} [Tue Aug 2 10:00:58 2022] PUT /identity/v3/system/users/b756d3dfe0ee4c10bebd90bc370b6c50/roles/0d0f0d7ac63143048ef3044aa9650882 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:58.850637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a834e-790a-463e-b99c-5424a93a6b92 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:58.987738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a834e-790a-463e-b99c-5424a93a6b92 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:59.027801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.028462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.028746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.029023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.040370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:00:59.045320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:00:59.046200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:00:59.047012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 164/331] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:00:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:00:59.052806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:59.151903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:59.185178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd848ee3-4e41-4d51-93d6-a02f7a94b3af None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:59.226734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.227457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.227744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.228019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.228809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:59.229122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:59.230502 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:59.242623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 168/332] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:00:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:59.247897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:59.269222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:59.356971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd59346-ac68-4cbf-8768-f97baebe8120 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:59.394085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8e532-9c42-4f67-8c45-2e3d0a73c523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.394828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8e532-9c42-4f67-8c45-2e3d0a73c523 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.395002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8e532-9c42-4f67-8c45-2e3d0a73c523 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.395290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8e532-9c42-4f67-8c45-2e3d0a73c523 None admin] PATH_INFO: `/v3/users/d88064b1f2294b30b02957189db19554` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.396467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8e532-9c42-4f67-8c45-2e3d0a73c523 None admin] RBAC: Authorizing `identity:get_user(user_id=d88064b1f2294b30b02957189db19554)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:59.398026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8e532-9c42-4f67-8c45-2e3d0a73c523 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:59.399884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 165/333] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:00:59 2022] GET /identity/v3/users/d88064b1f2294b30b02957189db19554 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:59.405676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:59.520469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20ff2d-d377-46a2-8b79-e53456aaf62b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:59.557877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d4fd3-8c5a-42fc-ac6e-c407497ffce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:59.558511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4fd3-8c5a-42fc-ac6e-c407497ffce9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:59.558798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4fd3-8c5a-42fc-ac6e-c407497ffce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:59.559077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4fd3-8c5a-42fc-ac6e-c407497ffce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:59.559906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d4fd3-8c5a-42fc-ac6e-c407497ffce9 None admin] RBAC: Adding query filter params (role.id=3c92f2e9a9cc4210be2f0e54650bab15, user.id=d88064b1f2294b30b02957189db19554, scope.system=all) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:59.560221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d4fd3-8c5a-42fc-ac6e-c407497ffce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:59.561657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d4fd3-8c5a-42fc-ac6e-c407497ffce9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:59.564413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:59.576418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 169/334] 10.222.0.49 () {56 vars in 1308 bytes} [Tue Aug 2 10:00:59 2022] GET /identity/v3/role_assignments?user.id=d88064b1f2294b30b02957189db19554&scope.system=all&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:01.018341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5adb6d-a0ae-4726-8251-8ad321b6ce8c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.019251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5adb6d-a0ae-4726-8251-8ad321b6ce8c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.019676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5adb6d-a0ae-4726-8251-8ad321b6ce8c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.021176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 166/335] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:01.035781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f71320-e08a-4681-b7cf-9efbb41f2620 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.036270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f71320-e08a-4681-b7cf-9efbb41f2620 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.036344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f71320-e08a-4681-b7cf-9efbb41f2620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.058833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-52f71320-e08a-4681-b7cf-9efbb41f2620 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:01.060059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-52f71320-e08a-4681-b7cf-9efbb41f2620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:01.151279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 170/336] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:01.189315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5adb6d-a0ae-4726-8251-8ad321b6ce8c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:01.313168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:01.335653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5adb6d-a0ae-4726-8251-8ad321b6ce8c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:01.375310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:01.376335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.376817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.377287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.389135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:01.394465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:01.395356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:01.396139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:01.398788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 167/337] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:01.404537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f71320-e08a-4681-b7cf-9efbb41f2620 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:01.549020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f71320-e08a-4681-b7cf-9efbb41f2620 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:01.577491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:01.590946 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7d753-fc6a-49fe-bd27-3c5f23d2e78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:01.591655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7d753-fc6a-49fe-bd27-3c5f23d2e78f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.591968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7d753-fc6a-49fe-bd27-3c5f23d2e78f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.592257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7d753-fc6a-49fe-bd27-3c5f23d2e78f None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.593015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7d753-fc6a-49fe-bd27-3c5f23d2e78f None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:01.597202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7d753-fc6a-49fe-bd27-3c5f23d2e78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:01.597202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7d753-fc6a-49fe-bd27-3c5f23d2e78f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:01.604845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 171/338] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:01.610730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:01.736837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819f63e-0384-40fd-b94d-e23325100a1a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:01.779088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:01.779782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.780072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.780354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.804951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:01.846635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634907-7c97-4b61-9e7e-a2c9fe8b9b4c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.846927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634907-7c97-4b61-9e7e-a2c9fe8b9b4c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.847146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634907-7c97-4b61-9e7e-a2c9fe8b9b4c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.848760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 172/339] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:01.863610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64d4a6-a983-49a0-9bec-218f34574d27 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.863920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64d4a6-a983-49a0-9bec-218f34574d27 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.864190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64d4a6-a983-49a0-9bec-218f34574d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.873245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:01.874805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:01.904310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ba64d4a6-a983-49a0-9bec-218f34574d27 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:01.905544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ba64d4a6-a983-49a0-9bec-218f34574d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.015941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 173/340] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:02.026155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 168/341] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:02.032236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64d4a6-a983-49a0-9bec-218f34574d27 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.078940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.147324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64d4a6-a983-49a0-9bec-218f34574d27 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.170759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.196805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.209021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754f2b4-9506-4e4d-8c11-9ed03ee4dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.210573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754f2b4-9506-4e4d-8c11-9ed03ee4dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.212196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754f2b4-9506-4e4d-8c11-9ed03ee4dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.212702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754f2b4-9506-4e4d-8c11-9ed03ee4dbb1 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.214291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754f2b4-9506-4e4d-8c11-9ed03ee4dbb1 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.216779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754f2b4-9506-4e4d-8c11-9ed03ee4dbb1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.219647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 174/342] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:02.221057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dd6c0-d580-482b-878f-876b3e334cab None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.221546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dd6c0-d580-482b-878f-876b3e334cab None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.222038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dd6c0-d580-482b-878f-876b3e334cab None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.223552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 175/343] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:02.224817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c028c5a-41dd-450a-8946-4d2d350ac5ef None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.225293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c028c5a-41dd-450a-8946-4d2d350ac5ef None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.226275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c028c5a-41dd-450a-8946-4d2d350ac5ef None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.231304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 176/344] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:02.233586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f9b62-6482-43a5-95b3-71d77578802f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.234278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f9b62-6482-43a5-95b3-71d77578802f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.234724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f9b62-6482-43a5-95b3-71d77578802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.255582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5b7f9b62-6482-43a5-95b3-71d77578802f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.257277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5b7f9b62-6482-43a5-95b3-71d77578802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.263805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70836-f8da-4132-801e-937c1bc6b6e2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.314590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a3822-e0c9-4d93-b83d-a64002dcfd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.315292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a3822-e0c9-4d93-b83d-a64002dcfd32 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.315583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a3822-e0c9-4d93-b83d-a64002dcfd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.315903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a3822-e0c9-4d93-b83d-a64002dcfd32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.331579 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a3822-e0c9-4d93-b83d-a64002dcfd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8e4a3822-e0c9-4d93-b83d-a64002dcfd32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:02.338970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:02.340048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:02.340857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 169/345] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:02.341574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391917-dcad-41f1-a173-fa648ec27a90 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.341794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391917-dcad-41f1-a173-fa648ec27a90 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.341987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391917-dcad-41f1-a173-fa648ec27a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.346865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 177/346] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:02.348061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f9b62-6482-43a5-95b3-71d77578802f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.376787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1d391917-dcad-41f1-a173-fa648ec27a90 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.378139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1d391917-dcad-41f1-a173-fa648ec27a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.474705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 170/347] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:02.475951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d391917-dcad-41f1-a173-fa648ec27a90 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.492183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f9b62-6482-43a5-95b3-71d77578802f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.515386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.524786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.552255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.552927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.553523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.553831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.563468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.565138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.567532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 178/348] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:02.568818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.625723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d391917-dcad-41f1-a173-fa648ec27a90 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.686964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2043c99-f481-4d8c-8e39-40f3db5beb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.687828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043c99-f481-4d8c-8e39-40f3db5beb9d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.688115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043c99-f481-4d8c-8e39-40f3db5beb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.688458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043c99-f481-4d8c-8e39-40f3db5beb9d None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.689499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2043c99-f481-4d8c-8e39-40f3db5beb9d None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:02.689935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2043c99-f481-4d8c-8e39-40f3db5beb9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.692286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2043c99-f481-4d8c-8e39-40f3db5beb9d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.699578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a013286-fe2d-4a19-a104-0af7dc562779 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.710042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 171/349] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:02.712457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc46d-0727-44cd-99f6-fe5ac528fe74 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.713284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc46d-0727-44cd-99f6-fe5ac528fe74 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.713829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc46d-0727-44cd-99f6-fe5ac528fe74 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.716855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 172/350] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:02.718175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfc46d-0727-44cd-99f6-fe5ac528fe74 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.743976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.744618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.744930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.745582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.765295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:02.771288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:02.772196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:02.773559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 179/351] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:02.774236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.833067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.870762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.880833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfc46d-0727-44cd-99f6-fe5ac528fe74 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.921948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.922603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.922886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.923172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.928181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f09b1-d3d4-4086-b029-ad68114af2fb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.938153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:02.946781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:02.947984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:02.948776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:02.949376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 173/352] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:02.952084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.972743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac4686-2109-414c-a863-e72e759483ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.973416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac4686-2109-414c-a863-e72e759483ed None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.973714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac4686-2109-414c-a863-e72e759483ed None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.974006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac4686-2109-414c-a863-e72e759483ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.974864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac4686-2109-414c-a863-e72e759483ed None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=37f4bbc3aa894e06971fb36f16623f07) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:02.975337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac4686-2109-414c-a863-e72e759483ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.976929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac4686-2109-414c-a863-e72e759483ed None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.984430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 180/353] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:02.986662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02a33-553a-450e-ac1c-65441d51934c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.987582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02a33-553a-450e-ac1c-65441d51934c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.988183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02a33-553a-450e-ac1c-65441d51934c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.019550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-33f02a33-553a-450e-ac1c-65441d51934c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:03.020799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-33f02a33-553a-450e-ac1c-65441d51934c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:03.089125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d27d0-7b76-4e4c-a9f9-e20f7db037a0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.145413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.145413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.145859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.146223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.158240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 181/354] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:03.159796 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f02a33-553a-450e-ac1c-65441d51934c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.189754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.198200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.256436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.258951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.305704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f02a33-553a-450e-ac1c-65441d51934c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.355827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.356459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.356750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.357047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.357784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:03.358199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.359739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.369885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 182/355] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.370778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.397361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 174/356] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:03.397361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.471358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.526903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.541427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753fcb-2b33-46f3-bc48-f64cb79b73a9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.541607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b223885-4c84-4f36-a1f6-e0f8659dc128 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.591531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c665904-80b7-4294-be38-0220e797737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.592121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c665904-80b7-4294-be38-0220e797737c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.592403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c665904-80b7-4294-be38-0220e797737c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.592675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c665904-80b7-4294-be38-0220e797737c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.596805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49320f-cb05-41d4-9d67-ff00e82a299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.597450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49320f-cb05-41d4-9d67-ff00e82a299c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.597742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49320f-cb05-41d4-9d67-ff00e82a299c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.598115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49320f-cb05-41d4-9d67-ff00e82a299c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.598822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49320f-cb05-41d4-9d67-ff00e82a299c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:03.599064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49320f-cb05-41d4-9d67-ff00e82a299c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.600675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49320f-cb05-41d4-9d67-ff00e82a299c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.606719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c665904-80b7-4294-be38-0220e797737c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:03.611161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 183/357] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.612438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4788a-2dd4-49d6-88aa-93154942bb51 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.612817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4788a-2dd4-49d6-88aa-93154942bb51 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.613164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4788a-2dd4-49d6-88aa-93154942bb51 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2c665904-80b7-4294-be38-0220e797737c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:03.614177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:03.615604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:03.617054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:03.617680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 184/358] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:03.617680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4788a-2dd4-49d6-88aa-93154942bb51 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.621101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 175/359] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:03.621697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c665904-80b7-4294-be38-0220e797737c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.777538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4788a-2dd4-49d6-88aa-93154942bb51 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.794251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c665904-80b7-4294-be38-0220e797737c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.820806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.821818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.822268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.823232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.823674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] PATH_INFO: `/v3/users/b756d3dfe0ee4c10bebd90bc370b6c50` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.825077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] RBAC: Authorizing `identity:get_user(user_id=b756d3dfe0ee4c10bebd90bc370b6c50)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.826697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.829472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 185/360] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/users/b756d3dfe0ee4c10bebd90bc370b6c50 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.833195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.850107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b086e-ee8a-4a1f-8ade-b63d0daf8d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.851520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b086e-ee8a-4a1f-8ade-b63d0daf8d04 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.851817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b086e-ee8a-4a1f-8ade-b63d0daf8d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.852097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b086e-ee8a-4a1f-8ade-b63d0daf8d04 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.853828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b086e-ee8a-4a1f-8ade-b63d0daf8d04 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.856244 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b086e-ee8a-4a1f-8ade-b63d0daf8d04 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.858342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 176/361] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:03.859590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf9ed0-a0bc-458c-b5ce-3d1a8e6c235e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.859883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf9ed0-a0bc-458c-b5ce-3d1a8e6c235e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.860171 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf9ed0-a0bc-458c-b5ce-3d1a8e6c235e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.868407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.889654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dccf9ed0-a0bc-458c-b5ce-3d1a8e6c235e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:03.890878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dccf9ed0-a0bc-458c-b5ce-3d1a8e6c235e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:03.966782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dc355-5fbb-46e8-9c1a-537f40efbe1c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.987223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 177/362] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:03.987223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf9ed0-a0bc-458c-b5ce-3d1a8e6c235e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.015987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.016751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.017177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.017478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.114708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.116481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.121636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf9ed0-a0bc-458c-b5ce-3d1a8e6c235e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.143965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.150261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.169197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.170654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.171520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.172288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.173245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:04.173724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.175136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.191309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 178/363] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:04.192206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.244595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 186/364] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:04.246108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.327301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13562d01-d660-40c7-bd8f-204aaf32f0a5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.369755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264a368-99a6-4cd7-b3e2-4412a467c10e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.375030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.375957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.376460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.377060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.378216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] RBAC: Adding query filter params (role.id=0d0f0d7ac63143048ef3044aa9650882, user.id=b756d3dfe0ee4c10bebd90bc370b6c50, scope.system=all) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:04.378684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.380877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.403484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 179/365] 10.222.0.49 () {56 vars in 1308 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/role_assignments?user.id=b756d3dfe0ee4c10bebd90bc370b6c50&scope.system=all&role.id=0d0f0d7ac63143048ef3044aa9650882 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:04.404641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.411228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.424342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.425200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.425654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.426040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.430696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.430696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.430990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 187/366] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:04.432608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.490191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.587621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f4009-12d9-474e-b153-2ca038c3b52d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.599495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435ab47-a002-4b23-a647-389bf3c05085 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.628254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.628879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.629171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.629454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.640024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.640693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.641085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.641368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.654292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:04.661038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:04.662408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 180/367] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:04.665544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.696599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.706154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.707692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.804823 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.813025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaae1d-4c6b-4f99-8707-3f371e5b55a1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.861908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.862584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.862915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.863155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.864544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.866403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.873920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 181/368] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:04.874078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 188/369] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:04.874078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.874486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.016353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.020627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dee755-ef8c-4a08-9b00-9b4c9ae25480 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.034132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aec2fe-9fea-4c83-a75e-d3aeee748402 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.071553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.072298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.072589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.072867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.073668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:05.073983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.078364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.081935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 182/370] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.082853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.088332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.089406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.089833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.090280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.091503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:05.091992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.094256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.108457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 189/371] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.109353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.130792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.244334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7282d9b-b262-4189-a489-b30b606fa083 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.256679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70247c-5f08-4fce-b1a5-604e8bc33517 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.286005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.286631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.286921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.287281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.288689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.290347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.299373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 183/372] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.301243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.307899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.308855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.309359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.310815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.313273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.314770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.316544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 190/373] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.322394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.351827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.417678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.423218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9efea8-8630-4fc7-85c9-7d238d4455bf None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.436059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2065e5-342f-4d0e-9759-eb31116d81b8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.470447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aad0d2-893e-4ce2-b09f-a978433347b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.471095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aad0d2-893e-4ce2-b09f-a978433347b7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.471525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aad0d2-893e-4ce2-b09f-a978433347b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.471928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aad0d2-893e-4ce2-b09f-a978433347b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.481895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.482812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.483654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.484108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.485378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=37f4bbc3aa894e06971fb36f16623f07) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:05.485862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.487512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.494635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 191/374] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.495696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.579490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aad0d2-893e-4ce2-b09f-a978433347b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.582155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aad0d2-893e-4ce2-b09f-a978433347b7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.613782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.644463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a3ca8-2242-4845-990f-ca29626bbf3d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.692240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25701dd-1c43-46cf-bc09-b4651c57238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.692910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25701dd-1c43-46cf-bc09-b4651c57238d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.693199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25701dd-1c43-46cf-bc09-b4651c57238d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.693483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25701dd-1c43-46cf-bc09-b4651c57238d None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.694681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25701dd-1c43-46cf-bc09-b4651c57238d None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.696197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25701dd-1c43-46cf-bc09-b4651c57238d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.701899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 192/375] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:05.701899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58323aa-979c-4e6f-8ed6-997145a3a515 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.701899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58323aa-979c-4e6f-8ed6-997145a3a515 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.701899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58323aa-979c-4e6f-8ed6-997145a3a515 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.701899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 193/376] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:05.706153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58323aa-979c-4e6f-8ed6-997145a3a515 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.731227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:05.735097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 184/377] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:05.736358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166b5cc-64d9-41be-a563-24e5b9f84618 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.736719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166b5cc-64d9-41be-a563-24e5b9f84618 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.737041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166b5cc-64d9-41be-a563-24e5b9f84618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.759364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8166b5cc-64d9-41be-a563-24e5b9f84618 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:05.761348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8166b5cc-64d9-41be-a563-24e5b9f84618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:05.837439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58323aa-979c-4e6f-8ed6-997145a3a515 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.850133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 185/378] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:05.851924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166b5cc-64d9-41be-a563-24e5b9f84618 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.898706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.899878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.900390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.900880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.902487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:05.902911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.904517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.911280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 194/379] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:05.915407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.961944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.071730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166b5cc-64d9-41be-a563-24e5b9f84618 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.134414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.148785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.149704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.150131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.150576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] PATH_INFO: `/v3/projects/737b03aeb0594b48a03adad9f790a347` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.151159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a67d4-d6b1-431a-816c-0c1c7eacbb13 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.152226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] RBAC: Authorizing `identity:get_project(project_id=737b03aeb0594b48a03adad9f790a347)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.154104 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.157842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 186/380] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/projects/737b03aeb0594b48a03adad9f790a347 => generated 344 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.165769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.222861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.224066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.225100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.252174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:06.258218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:06.259781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:06.260788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:06.271232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 195/381] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:06.275706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.361892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db3895-2e35-445c-858c-1539d313ce97 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.385126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.418649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41958c62-bce0-4dd7-beed-190b66a97c0b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.435705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.436405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.436706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.437005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] PATH_INFO: `/v3/groups/0bc76235028443fc8607cf951148e811` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.447905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] RBAC: Authorizing `identity:get_group(group_id=0bc76235028443fc8607cf951148e811)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.449493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.451319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 187/382] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/groups/0bc76235028443fc8607cf951148e811 => generated 229 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.457267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.495752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.496438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.496756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.497059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.497831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:06.498164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.499668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.523667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 196/383] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:06.526899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.585669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.665528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dd93a-0a67-48a3-aed2-8c512235d37c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.705061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.705711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.706087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.706403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.707277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] RBAC: Adding query filter params (group.id=0bc76235028443fc8607cf951148e811, role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=737b03aeb0594b48a03adad9f790a347) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:06.707621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.709157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.715561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 188/384] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/role_assignments?group.id=0bc76235028443fc8607cf951148e811&scope.project.id=737b03aeb0594b48a03adad9f790a347&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.731334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155bb0b-270e-4f28-ac9b-1579c2ccf7ce None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.789775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.790504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.790799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.791091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.792423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.794112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.796425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 197/385] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:06.804941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.954248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad10bd-381c-4522-9927-b65b8f300e48 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.981223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:06.995418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5683dc-e42b-4e63-adf1-ddccd177b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:07.001839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5683dc-e42b-4e63-adf1-ddccd177b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.001839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5683dc-e42b-4e63-adf1-ddccd177b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.001839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5683dc-e42b-4e63-adf1-ddccd177b3c6 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.001839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5683dc-e42b-4e63-adf1-ddccd177b3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:07.001839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5683dc-e42b-4e63-adf1-ddccd177b3c6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:07.001839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 189/386] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:07.008582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:07.047554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:07.155542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c9729-73e5-48b5-a912-3bd8ec03efc2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:07.205462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9a05a-db2c-438d-925d-d1b051460fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:07.206068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a05a-db2c-438d-925d-d1b051460fb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.206354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a05a-db2c-438d-925d-d1b051460fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.206627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a05a-db2c-438d-925d-d1b051460fb9 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07/groups/aba86117c4d145b1ba58eebd182edb82/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.209231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9a05a-db2c-438d-925d-d1b051460fb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=37f4bbc3aa894e06971fb36f16623f07, group_id=aba86117c4d145b1ba58eebd182edb82, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:07.213470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9a05a-db2c-438d-925d-d1b051460fb9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:07.284260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 198/387] 10.222.0.49 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:07 2022] PUT /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07/groups/aba86117c4d145b1ba58eebd182edb82/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:07.496796 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ed272-139c-4da7-a723-cf136f49f8f3 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.497396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ed272-139c-4da7-a723-cf136f49f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.497396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ed272-139c-4da7-a723-cf136f49f8f3 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.499221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 190/388] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:07.505095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06e79-9afd-4cb0-b0b8-71e8039642f1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.505428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06e79-9afd-4cb0-b0b8-71e8039642f1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.505746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06e79-9afd-4cb0-b0b8-71e8039642f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.537596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7ae06e79-9afd-4cb0-b0b8-71e8039642f1 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:07.539446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ae06e79-9afd-4cb0-b0b8-71e8039642f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:07.618926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:07.624418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 199/389] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:07.649579 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ed272-139c-4da7-a723-cf136f49f8f3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:07.796003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ed272-139c-4da7-a723-cf136f49f8f3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:07.842454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:07.843125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.843438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.843715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.865396 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0faef-798b-4257-a835-8a92d24a3d01 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.865751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0faef-798b-4257-a835-8a92d24a3d01 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.866038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0faef-798b-4257-a835-8a92d24a3d01 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.868359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 200/390] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:07.870979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:07.873343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0ec67-e26a-49a7-91cb-b210e9bd7d71 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:07.873632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0ec67-e26a-49a7-91cb-b210e9bd7d71 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:07.873906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0ec67-e26a-49a7-91cb-b210e9bd7d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:07.877943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:07.878955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:07.879831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:07.885884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 191/391] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:07.892815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:07.927332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-56d0ec67-e26a-49a7-91cb-b210e9bd7d71 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:07.928546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-56d0ec67-e26a-49a7-91cb-b210e9bd7d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:07.979921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:08.063289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ca91a-5f05-4585-b69f-47b04a18ade7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:08.113185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 201/392] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:08.119298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:08.119298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.119298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.119298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.119664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:08.119664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:08.119664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:08.135376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 192/393] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:08.137899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0ec67-e26a-49a7-91cb-b210e9bd7d71 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:08.149634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:08.362033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b5b0a-9bac-4b54-9d68-7fa44d820079 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:08.378656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0ec67-e26a-49a7-91cb-b210e9bd7d71 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:08.392809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:08.431973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:08.432894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:08.434071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.434377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.435574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.448575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:08.449232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.449512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.449790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.452820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:08.454606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:08.458994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 202/394] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:08 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:08.459559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:08.462781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:08.467618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:08.468469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:08.469371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:08.473410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 193/395] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:08.480391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:08.594931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50a4aa-3297-4326-b6b6-f44021ea8286 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:08.648679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17241c3-cbb1-41ca-a13a-cf0407b87d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:08.649758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17241c3-cbb1-41ca-a13a-cf0407b87d5b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.650263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17241c3-cbb1-41ca-a13a-cf0407b87d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.650785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17241c3-cbb1-41ca-a13a-cf0407b87d5b None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.652789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17241c3-cbb1-41ca-a13a-cf0407b87d5b None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:08.654728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17241c3-cbb1-41ca-a13a-cf0407b87d5b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:08.656677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 203/396] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:08 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:08.662645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9151765-ba95-4978-b281-14825448f944 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.663857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9151765-ba95-4978-b281-14825448f944 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.664402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9151765-ba95-4978-b281-14825448f944 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.665821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 204/397] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:08.668304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9151765-ba95-4978-b281-14825448f944 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:08.671364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d8645-e1f3-4941-a262-6f5a655d681c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:08.713520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94189de-2740-4f0d-af6a-937281011030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:08.714221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94189de-2740-4f0d-af6a-937281011030 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.715370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94189de-2740-4f0d-af6a-937281011030 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.715742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94189de-2740-4f0d-af6a-937281011030 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.716576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94189de-2740-4f0d-af6a-937281011030 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:08.716898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94189de-2740-4f0d-af6a-937281011030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:08.718299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94189de-2740-4f0d-af6a-937281011030 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:08.727735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 194/398] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:08.729049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba937a7-fc7f-4a10-802f-c227e738fef2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.729373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba937a7-fc7f-4a10-802f-c227e738fef2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.729686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba937a7-fc7f-4a10-802f-c227e738fef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.750694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6ba937a7-fc7f-4a10-802f-c227e738fef2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:08.759105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6ba937a7-fc7f-4a10-802f-c227e738fef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:08.759269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:08.780885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:08.797727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9151765-ba95-4978-b281-14825448f944 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:08.836131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 195/399] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:08.837914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba937a7-fc7f-4a10-802f-c227e738fef2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:08.848192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:08.849287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:08.849746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:08.850196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/groups/aba86117c4d145b1ba58eebd182edb82/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:08.857430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] RBAC: Authorizing `identity:create_grant(project_id=5895b755ae0c4e88a8d97321cf0b4411, group_id=aba86117c4d145b1ba58eebd182edb82, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:08.859137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:08.962766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 205/400] 10.222.0.49 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:08 2022] PUT /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/groups/aba86117c4d145b1ba58eebd182edb82/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:08.964208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:08.974207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba937a7-fc7f-4a10-802f-c227e738fef2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.046895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.049256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.053415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.054483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.056201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:09.058609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:09.063385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 196/401] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:08 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:09.066324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.129100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:09.201461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d1cf-08df-4010-ac1c-b92ba9cb7d94 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.224535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b74ea1-c87f-40e3-bcc2-049534e75c71 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.237559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:09.258005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.260181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.260845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.262032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.281955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:09.291838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:09.292761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:09.293582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:09.296689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 206/402] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:09.298962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.299850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.300344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.300784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.302348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:09.303361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.305025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:09.311274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 197/403] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:09 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:09.318698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.455878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8e7f8-601c-44b4-b94a-673b64b48ac9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.468206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:09.472305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61827584-8bd9-4c61-8859-55b99375de2f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.502751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844952-0c9f-4bc0-99be-dd36400ecb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.503415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844952-0c9f-4bc0-99be-dd36400ecb17 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.503698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844952-0c9f-4bc0-99be-dd36400ecb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.503971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844952-0c9f-4bc0-99be-dd36400ecb17 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.504854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844952-0c9f-4bc0-99be-dd36400ecb17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:09.505843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844952-0c9f-4bc0-99be-dd36400ecb17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:09.508891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844952-0c9f-4bc0-99be-dd36400ecb17 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:09.519644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 207/404] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:09.521265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a497b-a94a-4b20-a1d6-4752b2734c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.521686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c37a0-66df-4077-9a66-74c38c824d51 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.521951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a497b-a94a-4b20-a1d6-4752b2734c4c None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.522232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a497b-a94a-4b20-a1d6-4752b2734c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.522562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a497b-a94a-4b20-a1d6-4752b2734c4c None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07/groups/aba86117c4d145b1ba58eebd182edb82/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.523391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c37a0-66df-4077-9a66-74c38c824d51 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.523915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c37a0-66df-4077-9a66-74c38c824d51 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.525509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a497b-a94a-4b20-a1d6-4752b2734c4c None admin] RBAC: Authorizing `identity:create_grant(project_id=37f4bbc3aa894e06971fb36f16623f07, group_id=aba86117c4d145b1ba58eebd182edb82, role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:09.525964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 208/405] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:09.526873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c37a0-66df-4077-9a66-74c38c824d51 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.535739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a497b-a94a-4b20-a1d6-4752b2734c4c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:09.579609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:09.634316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c37a0-66df-4077-9a66-74c38c824d51 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.645988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 198/406] 10.222.0.49 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:09 2022] PUT /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07/groups/aba86117c4d145b1ba58eebd182edb82/roles/9a49213a53964452a411e2dbd74d7c5e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:09.654210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054cdb1-a207-4a2b-a1fa-c735842c7d68 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.654210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054cdb1-a207-4a2b-a1fa-c735842c7d68 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.654210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054cdb1-a207-4a2b-a1fa-c735842c7d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.672100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f054cdb1-a207-4a2b-a1fa-c735842c7d68 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:09.677233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f054cdb1-a207-4a2b-a1fa-c735842c7d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:09.690659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.692059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.693127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.693978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.696679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:09.698846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:09.706893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 209/407] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:09 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:09.708456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.804576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 199/408] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:09.806519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef9d6-5b4d-4698-bbfd-2798d8cf4aea None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.807024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef9d6-5b4d-4698-bbfd-2798d8cf4aea None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.807537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef9d6-5b4d-4698-bbfd-2798d8cf4aea None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.809136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 200/409] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:09.816255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7676ff-2192-4cd0-8be1-1cbac6179d06 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.816917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7676ff-2192-4cd0-8be1-1cbac6179d06 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.817456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7676ff-2192-4cd0-8be1-1cbac6179d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.837939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7c7676ff-2192-4cd0-8be1-1cbac6179d06 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:09.839624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c7676ff-2192-4cd0-8be1-1cbac6179d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:09.850710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba82ab9-6837-453e-ba4e-2d68c8211b98 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:09.867477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:09.889907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:09.891028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:09.891028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:09.891372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:09.892618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:09.894088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:09.895965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 210/410] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:09 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:09.897848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.934057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 201/411] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:09.934865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7676ff-2192-4cd0-8be1-1cbac6179d06 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:09.966307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.030855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cfcb8-7c98-4a97-9b88-1146782c5e65 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.070499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.071102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.071464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.071748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.076727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7676ff-2192-4cd0-8be1-1cbac6179d06 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.083620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:10.089517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:10.090384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:10.091168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 211/412] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:10.091858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.126759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.127474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.127778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.128155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/groups/aba86117c4d145b1ba58eebd182edb82/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.130767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] RBAC: Authorizing `identity:create_grant(project_id=5895b755ae0c4e88a8d97321cf0b4411, group_id=aba86117c4d145b1ba58eebd182edb82, role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:10.132352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:10.145809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.204922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e750713-8b49-4897-86e1-12626b2a02fe None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.214500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 202/413] 10.222.0.49 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:09 2022] PUT /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411/groups/aba86117c4d145b1ba58eebd182edb82/roles/9a49213a53964452a411e2dbd74d7c5e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:10.214500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.276018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.277836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.278134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.278414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.304368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:10.312699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:10.313662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:10.314913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 212/414] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:10.324135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.419649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.441267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342ed14-b0ea-492e-8f6f-4b46ffc3a875 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.458645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937c863-97ca-4535-a0b2-d30fa4734917 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.467438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.484596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.485352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.485708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.486056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.486847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:10.487445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:10.489641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:10.500958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 203/415] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:10.506324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.511460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.511460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.511460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.511460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.511810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:10.511810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:10.511810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:10.523177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 213/416] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:10.529426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.649230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d682d3-68a2-4da2-ba9c-2b119e60661b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.696960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.697620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.697885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.698192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] PATH_INFO: `/v3/groups/0bc76235028443fc8607cf951148e811` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.700960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc76235028443fc8607cf951148e811)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:10.702412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:10.704048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 204/417] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/groups/0bc76235028443fc8607cf951148e811 => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:10.709719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:10.734120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.776981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373def-ec5e-464c-afc4-e9ba2519e20c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.844052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.845069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.845069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.845247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.876002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcfe81-6aa2-4778-b1f5-d53baac9caa1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:10.914630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:10.915869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:10.916511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:10.916797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:10.917092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] PATH_INFO: `/v3/projects/737b03aeb0594b48a03adad9f790a347` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:10.918397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] RBAC: Authorizing `identity:get_project(project_id=737b03aeb0594b48a03adad9f790a347)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:10.920202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:10.922214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 205/418] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/projects/737b03aeb0594b48a03adad9f790a347 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:10.928031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.011855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.014075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.043596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.087595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6338f38-ccb0-423a-bacf-640310388d5e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.152868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385503af-2da1-4f45-a70c-2b3a749b7245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.153629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385503af-2da1-4f45-a70c-2b3a749b7245 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.153921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385503af-2da1-4f45-a70c-2b3a749b7245 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.154201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385503af-2da1-4f45-a70c-2b3a749b7245 None admin] PATH_INFO: `/v3/projects/737b03aeb0594b48a03adad9f790a347/groups/0bc76235028443fc8607cf951148e811/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.156223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385503af-2da1-4f45-a70c-2b3a749b7245 None admin] RBAC: Authorizing `identity:create_grant(project_id=737b03aeb0594b48a03adad9f790a347, group_id=0bc76235028443fc8607cf951148e811, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.157750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385503af-2da1-4f45-a70c-2b3a749b7245 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.182389 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 214/419] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:11.188277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.238008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 206/420] 10.222.0.49 () {58 vars in 1442 bytes} [Tue Aug 2 10:01:10 2022] PUT /identity/v3/projects/737b03aeb0594b48a03adad9f790a347/groups/0bc76235028443fc8607cf951148e811/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:11.252001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47250-e440-4b81-bb40-56c2e0a14f7c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.252397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47250-e440-4b81-bb40-56c2e0a14f7c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.252602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47250-e440-4b81-bb40-56c2e0a14f7c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.258249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 207/421] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:11.266872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace7c7-5d2c-4cda-9e7c-032b35d3f096 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.267226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace7c7-5d2c-4cda-9e7c-032b35d3f096 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.267482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace7c7-5d2c-4cda-9e7c-032b35d3f096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.271614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.298186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-35ace7c7-5d2c-4cda-9e7c-032b35d3f096 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:11.307400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412f8b4-f29e-4e67-8286-4ee0fadff288 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.307652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-35ace7c7-5d2c-4cda-9e7c-032b35d3f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:11.345113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.345768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.346054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.346333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.347545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.349054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.351628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 215/422] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:11.361202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.447070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 208/423] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:11.472886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ace7c7-5d2c-4cda-9e7c-032b35d3f096 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.521948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdece5-7c4f-41c8-b751-a496f2a750ef None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.592763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.598310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.601615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.605895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.605895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.605895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.605895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.605895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.613825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 216/424] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:11.620169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.654250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ace7c7-5d2c-4cda-9e7c-032b35d3f096 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.720900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.721505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.721753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.722034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.748187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:11.753225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:11.755555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:11.756324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 209/425] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:11.767294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.791451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c026b-7546-4a8b-b416-6118f97494b3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.831206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.832249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.832711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.833146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.834129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=37f4bbc3aa894e06971fb36f16623f07) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:11.834613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.838059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.845786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 217/426] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:11.936650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f1771-9bd8-4c92-b6d4-87318c0e8f59 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.970073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.983611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3de43-7d37-404a-b163-809c6c61936f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.983611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3de43-7d37-404a-b163-809c6c61936f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.983611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3de43-7d37-404a-b163-809c6c61936f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.983611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3de43-7d37-404a-b163-809c6c61936f None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.984080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3de43-7d37-404a-b163-809c6c61936f None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:11.984080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3de43-7d37-404a-b163-809c6c61936f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.985118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3de43-7d37-404a-b163-809c6c61936f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.996730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 210/427] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:12.004415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.084124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.124779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3357-6d3b-4d62-9e1a-d69cbf3b1b41 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.162759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.164049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.164336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.164609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.219500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444518-791b-497d-9de9-18176c438d29 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.219848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444518-791b-497d-9de9-18176c438d29 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.220052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444518-791b-497d-9de9-18176c438d29 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.225642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 211/428] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:12.227322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc757e-f9ac-4b2a-baf4-3d431c0e7905 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.227610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc757e-f9ac-4b2a-baf4-3d431c0e7905 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.227888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc757e-f9ac-4b2a-baf4-3d431c0e7905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.249981 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-17fc757e-f9ac-4b2a-baf4-3d431c0e7905 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:12.251183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-17fc757e-f9ac-4b2a-baf4-3d431c0e7905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:12.255009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.256617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.324721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.326226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 212/429] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:12.361522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc757e-f9ac-4b2a-baf4-3d431c0e7905 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.378136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 218/430] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:12.383950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.494990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc757e-f9ac-4b2a-baf4-3d431c0e7905 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.513496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91edc4-f698-49ce-bd71-3be450fdb3b6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.524619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.546014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7ce6d-51f8-4e38-b570-04c03947c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.547314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7ce6d-51f8-4e38-b570-04c03947c90b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.547785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7ce6d-51f8-4e38-b570-04c03947c90b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.548417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7ce6d-51f8-4e38-b570-04c03947c90b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.561407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7ce6d-51f8-4e38-b570-04c03947c90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:12.562878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.563651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.563932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.564208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.566402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.567389 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2fd7ce6d-51f8-4e38-b570-04c03947c90b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:12.567487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:12.568315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:12.569078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 213/431] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:12.570783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb919-2f76-4316-a01e-75750d31a2dc None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.570892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb919-2f76-4316-a01e-75750d31a2dc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.571124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb919-2f76-4316-a01e-75750d31a2dc None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.575232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 219/432] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:12.575362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 214/433] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:12.575454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.580241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161eb919-2f76-4316-a01e-75750d31a2dc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.612939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.695906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3377be6-660f-4b94-bb0d-3cad7cb4c732 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.708156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161eb919-2f76-4316-a01e-75750d31a2dc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.736326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13155ebe-5faf-40e5-862e-676bf48afaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.737073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13155ebe-5faf-40e5-862e-676bf48afaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.737376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13155ebe-5faf-40e5-862e-676bf48afaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.737746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13155ebe-5faf-40e5-862e-676bf48afaa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.738468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13155ebe-5faf-40e5-862e-676bf48afaa7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:12.738783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13155ebe-5faf-40e5-862e-676bf48afaa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.740623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13155ebe-5faf-40e5-862e-676bf48afaa7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.749325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.755299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 220/434] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:12.755299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2d301-8f4e-4e3f-9e73-4db805b13868 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.755299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2d301-8f4e-4e3f-9e73-4db805b13868 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.755299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2d301-8f4e-4e3f-9e73-4db805b13868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.755609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.755609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.755609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.755609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.755609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.761870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 215/435] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:12.761870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.770609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e0a2d301-8f4e-4e3f-9e73-4db805b13868 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:12.771796 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e0a2d301-8f4e-4e3f-9e73-4db805b13868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:12.813103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.851247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 221/436] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:12.851247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2d301-8f4e-4e3f-9e73-4db805b13868 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.877466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.881746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d7ae-2e4f-4493-ad20-86cd0a1376b0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.917611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.918237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.918520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.918799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.964750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2d301-8f4e-4e3f-9e73-4db805b13868 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.993878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.995395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.013582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.014255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.014562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.014846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.015733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=3c92f2e9a9cc4210be2f0e54650bab15, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:13.016071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.017607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.024069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 222/437] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=3c92f2e9a9cc4210be2f0e54650bab15 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.024911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.068371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.131739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.146328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c62b6-b43e-40fd-8cb1-633d1eae3a86 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.157288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 216/438] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:13.160914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.197763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.198916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.199458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.199940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.215101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:13.221710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:13.222603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:13.223431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:13.229281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 223/439] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:13.231304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.282608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36735c-aa5c-4649-8b57-eb270c96597d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.318218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.318871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.319155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.319456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] PATH_INFO: `/v3/projects/37f4bbc3aa894e06971fb36f16623f07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.320865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] RBAC: Authorizing `identity:get_project(project_id=37f4bbc3aa894e06971fb36f16623f07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.322832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.324758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 217/440] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/projects/37f4bbc3aa894e06971fb36f16623f07 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.329835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.351312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.351579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf5bad-25cb-4c37-9818-59549a853397 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.392580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.393216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.393494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.393772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.394489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:13.394800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.396222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.404730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 224/441] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:13.409942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.422294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.443008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ce634-d56e-4c6d-b780-3b87b4a948d4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.481819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.482483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.482771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.483051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.484253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.485708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.487487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 218/442] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:13.493458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.515612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e652698-0d3c-4d7f-af28-dc1bb47060c2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.555005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.555654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.555937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.556213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.587468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.603694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8f1b8-aade-47bd-bdba-0c75fb7d8326 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.628004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.629475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.640648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf067e-af0a-4c7b-bbcc-c46e21293903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.641278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf067e-af0a-4c7b-bbcc-c46e21293903 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.641570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf067e-af0a-4c7b-bbcc-c46e21293903 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.641872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf067e-af0a-4c7b-bbcc-c46e21293903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.642042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.643066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf067e-af0a-4c7b-bbcc-c46e21293903 None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=37f4bbc3aa894e06971fb36f16623f07) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:13.643436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf067e-af0a-4c7b-bbcc-c46e21293903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.644974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf067e-af0a-4c7b-bbcc-c46e21293903 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.654964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 219/443] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=37f4bbc3aa894e06971fb36f16623f07&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:13.656237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67bdf4-d529-44ea-9824-81dc58648a3c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.656527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67bdf4-d529-44ea-9824-81dc58648a3c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.656815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67bdf4-d529-44ea-9824-81dc58648a3c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.658049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 220/444] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:13.664247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990871-c36f-4dc5-984f-555dc71e363f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.664547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990871-c36f-4dc5-984f-555dc71e363f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.664839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990871-c36f-4dc5-984f-555dc71e363f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.684158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d8990871-c36f-4dc5-984f-555dc71e363f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:13.685397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d8990871-c36f-4dc5-984f-555dc71e363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:13.766239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 225/445] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:01:13.767468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.771499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 221/446] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:13.796380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990871-c36f-4dc5-984f-555dc71e363f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.886686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.886897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccb2af-8ede-4904-b99d-348ebb963fc1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.898832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:13.911886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990871-c36f-4dc5-984f-555dc71e363f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.926013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.926889 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.927024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.927291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] PATH_INFO: `/v3/projects/5895b755ae0c4e88a8d97321cf0b4411` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.928435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5895b755ae0c4e88a8d97321cf0b4411)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.930427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.932144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 226/447] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/projects/5895b755ae0c4e88a8d97321cf0b4411 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:13.938331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.953742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.953936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.954094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.954379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.966980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:13.971987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:13.973048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:13.973879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 222/448] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:13.980702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.062941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4dcf0a-fa9b-4bc9-8348-765a56e808a6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.101288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b411294-6709-4faf-a458-93463c68488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.102266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b411294-6709-4faf-a458-93463c68488a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.102548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b411294-6709-4faf-a458-93463c68488a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.102824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b411294-6709-4faf-a458-93463c68488a None admin] PATH_INFO: `/v3/groups/aba86117c4d145b1ba58eebd182edb82` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.104114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b411294-6709-4faf-a458-93463c68488a None admin] RBAC: Authorizing `identity:get_group(group_id=aba86117c4d145b1ba58eebd182edb82)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.105569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b411294-6709-4faf-a458-93463c68488a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.107302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 227/449] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/groups/aba86117c4d145b1ba58eebd182edb82 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:14.110329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d65c7-3cb6-4432-9623-ca4e23f15249 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.113178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b411294-6709-4faf-a458-93463c68488a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.150641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.151408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.151695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.151971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.152694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:14.153000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.154369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.161072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.164639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 223/450] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:14.166295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.170252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.218657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b411294-6709-4faf-a458-93463c68488a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.253421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.254034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.254313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.254584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.255399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] RBAC: Adding query filter params (group.id=aba86117c4d145b1ba58eebd182edb82, role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=5895b755ae0c4e88a8d97321cf0b4411) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:14.255705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.257746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.263830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 228/451] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/role_assignments?group.id=aba86117c4d145b1ba58eebd182edb82&scope.project.id=5895b755ae0c4e88a8d97321cf0b4411&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:14.271005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45f5bd-dc93-4087-92c6-154391d342d0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.307074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.307737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.308021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.308296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.377761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.380164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.381795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.484206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 224/452] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:01:14.490739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.600269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8851168-91ba-4cef-9dfb-d9942c5144ab None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.610776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.634315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.634898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.635277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.635564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] PATH_INFO: `/v3/projects/737b03aeb0594b48a03adad9f790a347` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.636717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] RBAC: Authorizing `identity:get_project(project_id=737b03aeb0594b48a03adad9f790a347)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.638139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.639842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 229/453] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/projects/737b03aeb0594b48a03adad9f790a347 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:14.645503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.739957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:14.743287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3b75-59ac-4a5b-ba61-a1ebc6512e35 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.779243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5609cec-974f-4721-b3a3-7a205f1170ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.779937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5609cec-974f-4721-b3a3-7a205f1170ef None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.780251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5609cec-974f-4721-b3a3-7a205f1170ef None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.780583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5609cec-974f-4721-b3a3-7a205f1170ef None admin] PATH_INFO: `/v3/groups/0bc76235028443fc8607cf951148e811` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.781893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5609cec-974f-4721-b3a3-7a205f1170ef None admin] RBAC: Authorizing `identity:get_group(group_id=0bc76235028443fc8607cf951148e811)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.783420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5609cec-974f-4721-b3a3-7a205f1170ef None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.785054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 225/454] 10.222.0.49 () {56 vars in 1180 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/groups/0bc76235028443fc8607cf951148e811 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:14.791154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.888817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058aa2-6a78-4ad8-8767-932f1ff49088 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:14.920230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7f369-ba3a-4101-9b36-5bf4a72a3a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:14.920835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7f369-ba3a-4101-9b36-5bf4a72a3a2a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:14.921119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7f369-ba3a-4101-9b36-5bf4a72a3a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:14.921398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7f369-ba3a-4101-9b36-5bf4a72a3a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:14.922177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7f369-ba3a-4101-9b36-5bf4a72a3a2a None admin] RBAC: Adding query filter params (group.id=0bc76235028443fc8607cf951148e811, role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=737b03aeb0594b48a03adad9f790a347) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:14.922491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7f369-ba3a-4101-9b36-5bf4a72a3a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:14.923939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7f369-ba3a-4101-9b36-5bf4a72a3a2a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:14.930038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 230/455] 10.222.0.49 () {56 vars in 1376 bytes} [Tue Aug 2 10:01:14 2022] GET /identity/v3/role_assignments?group.id=0bc76235028443fc8607cf951148e811&scope.project.id=737b03aeb0594b48a03adad9f790a347&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:17.314213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf2b8-426e-499d-a862-d41676e92044 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.315243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf2b8-426e-499d-a862-d41676e92044 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.315732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf2b8-426e-499d-a862-d41676e92044 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.317562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 226/456] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:17.324023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2883c-83ee-4435-aa24-941ebf6f8a30 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.324836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2883c-83ee-4435-aa24-941ebf6f8a30 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.325314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2883c-83ee-4435-aa24-941ebf6f8a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.354034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5cd2883c-83ee-4435-aa24-941ebf6f8a30 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:17.357300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5cd2883c-83ee-4435-aa24-941ebf6f8a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:17.411517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.451303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 231/457] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:17.468732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf2b8-426e-499d-a862-d41676e92044 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.577897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf2b8-426e-499d-a862-d41676e92044 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.613898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57379d2-f8a3-4093-9a0d-5c3e7bef606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.614550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57379d2-f8a3-4093-9a0d-5c3e7bef606c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.614831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57379d2-f8a3-4093-9a0d-5c3e7bef606c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.615105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57379d2-f8a3-4093-9a0d-5c3e7bef606c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.626514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57379d2-f8a3-4093-9a0d-5c3e7bef606c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.627996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57379d2-f8a3-4093-9a0d-5c3e7bef606c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a57379d2-f8a3-4093-9a0d-5c3e7bef606c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:17.644029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:17.644991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:17.645891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:17.645891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:17.645891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:17.645891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:17.650033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 227/458] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:17.651448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2883c-83ee-4435-aa24-941ebf6f8a30 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.748700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b8bc-294e-4cc9-a799-d21da720c39a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.750271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b8bc-294e-4cc9-a799-d21da720c39a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.750271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b8bc-294e-4cc9-a799-d21da720c39a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.750746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 228/459] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:17.754442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b9848-38f1-4f49-9602-a2b4ab26ed68 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.755127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b9848-38f1-4f49-9602-a2b4ab26ed68 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.755442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b9848-38f1-4f49-9602-a2b4ab26ed68 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.757792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 229/460] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:17.758293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2883c-83ee-4435-aa24-941ebf6f8a30 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:17.758832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd54ef-3c70-4fa0-aed9-6d0a73885d3f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.759114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd54ef-3c70-4fa0-aed9-6d0a73885d3f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.759410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd54ef-3c70-4fa0-aed9-6d0a73885d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.782518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8bdd54ef-3c70-4fa0-aed9-6d0a73885d3f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:17.783860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8bdd54ef-3c70-4fa0-aed9-6d0a73885d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:17.807825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aaade-1419-4f48-b3b1-fbc589d2ece8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:17.808613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aaade-1419-4f48-b3b1-fbc589d2ece8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.809215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aaade-1419-4f48-b3b1-fbc589d2ece8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.809490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aaade-1419-4f48-b3b1-fbc589d2ece8 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.810461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aaade-1419-4f48-b3b1-fbc589d2ece8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:17.810769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aaade-1419-4f48-b3b1-fbc589d2ece8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:17.812973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aaade-1419-4f48-b3b1-fbc589d2ece8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:17.813123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.831353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 232/461] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:17.834251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2699-9f05-4ca5-9b27-68bf6971db63 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:17.835991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2699-9f05-4ca5-9b27-68bf6971db63 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:17.836290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2699-9f05-4ca5-9b27-68bf6971db63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:17.859372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-30be2699-9f05-4ca5-9b27-68bf6971db63 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:17.860895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-30be2699-9f05-4ca5-9b27-68bf6971db63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:17.862363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 230/462] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:17.863311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd54ef-3c70-4fa0-aed9-6d0a73885d3f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.896137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:17.948480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 233/463] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:17.956178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2699-9f05-4ca5-9b27-68bf6971db63 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:17.988900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd54ef-3c70-4fa0-aed9-6d0a73885d3f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.039404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.040682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.041004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.041309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.042265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.043796 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.082627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2699-9f05-4ca5-9b27-68bf6971db63 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.119482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.120276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.120567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.120836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.131702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.133792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:18.155452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:18.156379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:18.157278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:18.157278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:18.157278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:18.157278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:18.157278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 234/465] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:18.157278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.157653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 231/465] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:17 2022] POST /identity/v3/users => generated 246 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:18.157653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.181774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.187373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.290078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2fae7-c9ec-49c5-ad95-78742509e565 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.320091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d057b-303b-4449-a9ef-a301093003ce None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.332500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b845d-716e-4235-b40d-54894907a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.333181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b845d-716e-4235-b40d-54894907a63c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.333412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b845d-716e-4235-b40d-54894907a63c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.333689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b845d-716e-4235-b40d-54894907a63c None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.334756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b845d-716e-4235-b40d-54894907a63c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:18.335068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b845d-716e-4235-b40d-54894907a63c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.336984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b845d-716e-4235-b40d-54894907a63c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.363227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 232/466] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:18.367259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a78c71-f392-4e18-9a4d-8836a5162d4d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.367259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a78c71-f392-4e18-9a4d-8836a5162d4d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.367259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a78c71-f392-4e18-9a4d-8836a5162d4d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.367547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef8266-8eda-49af-b521-81968dd65ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.367547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef8266-8eda-49af-b521-81968dd65ddb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.367547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef8266-8eda-49af-b521-81968dd65ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.368130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef8266-8eda-49af-b521-81968dd65ddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.368574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 233/467] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:18.369635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a78c71-f392-4e18-9a4d-8836a5162d4d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.380916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef8266-8eda-49af-b521-81968dd65ddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.382989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef8266-8eda-49af-b521-81968dd65ddb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e5ef8266-8eda-49af-b521-81968dd65ddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:18.398910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:18.399928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:18.400757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:18.400757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:18.400757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:18.400757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:18.401449 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 235/468] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:18.402259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b779d1-4430-4524-a05e-b01cec8108ca None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.402564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b779d1-4430-4524-a05e-b01cec8108ca None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.402869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b779d1-4430-4524-a05e-b01cec8108ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.423926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c7b779d1-4430-4524-a05e-b01cec8108ca None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:18.425360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c7b779d1-4430-4524-a05e-b01cec8108ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:18.463321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.499091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.512877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a78c71-f392-4e18-9a4d-8836a5162d4d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.515065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 236/469] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:18.516141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b779d1-4430-4524-a05e-b01cec8108ca None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.556100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.556890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.557311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.557964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.559276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.561234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.636388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b779d1-4430-4524-a05e-b01cec8108ca None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.641711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 234/470] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:18 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:18.642650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.706006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.706944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.707446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.707874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.708739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:18.709200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.711035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.729706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 237/471] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:18.740406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.814663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73e9e5-b221-4e9a-b6ab-fca47fe198f6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.843573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.855501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:18.868219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb3909-e9f6-47ae-81d1-322785351690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.868911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb3909-e9f6-47ae-81d1-322785351690 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.869624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb3909-e9f6-47ae-81d1-322785351690 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.869908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb3909-e9f6-47ae-81d1-322785351690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.887773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb3909-e9f6-47ae-81d1-322785351690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.889779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb3909-e9f6-47ae-81d1-322785351690 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:18.899306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2886c-138b-4453-8ccb-c07972457cb7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-7afb3909-e9f6-47ae-81d1-322785351690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:18.908878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:18.909761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:18.910612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:18.910612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:18.910612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:18.910612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:18.910612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 235/472] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:18.911776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52927e4a-b8f4-4dbe-930e-6776d49ae4a4 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.911964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52927e4a-b8f4-4dbe-930e-6776d49ae4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.912254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52927e4a-b8f4-4dbe-930e-6776d49ae4a4 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.914093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 236/473] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:18.919380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52927e4a-b8f4-4dbe-930e-6776d49ae4a4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:18.948037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21accdd4-02ec-44c9-87bb-f72fc40923f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:18.948733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accdd4-02ec-44c9-87bb-f72fc40923f2 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:18.949013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accdd4-02ec-44c9-87bb-f72fc40923f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:18.949301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accdd4-02ec-44c9-87bb-f72fc40923f2 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:18.950063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21accdd4-02ec-44c9-87bb-f72fc40923f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:18.951611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21accdd4-02ec-44c9-87bb-f72fc40923f2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.056023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 238/474] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:18 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:19.057461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69657d-bf49-4fe6-805e-7f06b4daee62 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.057756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69657d-bf49-4fe6-805e-7f06b4daee62 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.058060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69657d-bf49-4fe6-805e-7f06b4daee62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.066100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52927e4a-b8f4-4dbe-930e-6776d49ae4a4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.092264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cd69657d-bf49-4fe6-805e-7f06b4daee62 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:19.093543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cd69657d-bf49-4fe6-805e-7f06b4daee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:19.142059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.142624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.142912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.143236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.143982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:19.144297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.145822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.175295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 237/475] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:19.177322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.242575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.246024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 239/476] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:19.268308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69657d-bf49-4fe6-805e-7f06b4daee62 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.326058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.340425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269a3b1-084a-4838-95f9-f06f02503f0a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.384782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.385905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.385905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.386143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.386842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.388671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.403277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69657d-bf49-4fe6-805e-7f06b4daee62 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.445853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.446487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.446772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.447047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.447846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.449236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.484678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.sql.core [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:01:19.484678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 10:01:19.484678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [parameters: {'id': 'dd435c5f33784f8ba3dc1414938ed216', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 10:01:19.484678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80466) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:01:19.492437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 238/477] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:19 2022] POST /identity/v3/users => generated 249 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() Aug 02 10:01:19.504364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return ref.to_dict() Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:01:19.505300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application yield resource Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise self.value Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application session.commit() Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:01:19.506187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:01:19.507011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:01:19.509643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:01:19.510640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [parameters: {'id': 'dd435c5f33784f8ba3dc1414938ed216', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:19.511692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 10:01:19.512627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:19.513644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 240/478] 10.222.0.49 () {60 vars in 1131 bytes} [Tue Aug 2 10:01:19 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:19.520397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.669736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.680400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d22b23-0405-4c49-a256-cdfecc074f1c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.719350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217926d5-c4bd-478a-a703-d4c99549fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.720000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217926d5-c4bd-478a-a703-d4c99549fa07 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.720397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217926d5-c4bd-478a-a703-d4c99549fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.720680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217926d5-c4bd-478a-a703-d4c99549fa07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.732328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217926d5-c4bd-478a-a703-d4c99549fa07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-217926d5-c4bd-478a-a703-d4c99549fa07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:19.737269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:19.738139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:19.738938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 239/479] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:19.748276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:19.858590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4748a-b63f-48c9-a177-fa19fcaaacf3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:19.867591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:19.897234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6745-5930-446c-b81e-583f1c18296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:19.897861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6745-5930-446c-b81e-583f1c18296a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:19.898171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6745-5930-446c-b81e-583f1c18296a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:19.898443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6745-5930-446c-b81e-583f1c18296a None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:19.899155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6745-5930-446c-b81e-583f1c18296a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:19.899493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6745-5930-446c-b81e-583f1c18296a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:19.900858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6745-5930-446c-b81e-583f1c18296a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:19.910159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 241/480] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:20.555038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2c12-53dc-42c3-a0ca-a5f8ee03a66b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:20.556019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2c12-53dc-42c3-a0ca-a5f8ee03a66b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:20.556491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2c12-53dc-42c3-a0ca-a5f8ee03a66b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:20.558296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 240/481] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:20.563318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc380-a43c-49ad-9dff-2b739649f399 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:20.564147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc380-a43c-49ad-9dff-2b739649f399 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:20.564629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc380-a43c-49ad-9dff-2b739649f399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:20.585483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f8ecc380-a43c-49ad-9dff-2b739649f399 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:20.587613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f8ecc380-a43c-49ad-9dff-2b739649f399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:20.672570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 242/482] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:20.707214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af2c12-53dc-42c3-a0ca-a5f8ee03a66b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:20.849019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af2c12-53dc-42c3-a0ca-a5f8ee03a66b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:20.900515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:20.901179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:20.901472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:20.901761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:20.916713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:20.922368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:20.923254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:20.924042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 241/483] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:20.932272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc380-a43c-49ad-9dff-2b739649f399 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.101312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.106944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc380-a43c-49ad-9dff-2b739649f399 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.173010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ea8b9-0679-4e21-8d0f-1598217ee926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.173643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ea8b9-0679-4e21-8d0f-1598217ee926 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.173930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ea8b9-0679-4e21-8d0f-1598217ee926 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.174218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ea8b9-0679-4e21-8d0f-1598217ee926 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.174948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ea8b9-0679-4e21-8d0f-1598217ee926 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:21.175285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ea8b9-0679-4e21-8d0f-1598217ee926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.176674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ea8b9-0679-4e21-8d0f-1598217ee926 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.186338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 243/484] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:21.192061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.216278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.258054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839d75b-652d-42b9-bf78-9caf4f66b6af None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.258361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839d75b-652d-42b9-bf78-9caf4f66b6af None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.258611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839d75b-652d-42b9-bf78-9caf4f66b6af None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.260159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 244/485] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:21.267519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3bfd8-4939-44e9-aeb8-2f4a0d9e90b7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.267835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3bfd8-4939-44e9-aeb8-2f4a0d9e90b7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.268183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3bfd8-4939-44e9-aeb8-2f4a0d9e90b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.297959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9ae3bfd8-4939-44e9-aeb8-2f4a0d9e90b7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:21.299206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9ae3bfd8-4939-44e9-aeb8-2f4a0d9e90b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:21.343339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a279-0cec-4d45-ac60-4b0987fdc9a6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.375077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 245/486] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:21.399281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3bfd8-4939-44e9-aeb8-2f4a0d9e90b7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.411918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.412589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.412884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.413163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.424570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.426056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:21.453123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:21.455155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:21.457213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:21.457213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:21.457213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:21.457213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:21.457213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 242/487] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:21.465052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.504917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.515495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3bfd8-4939-44e9-aeb8-2f4a0d9e90b7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.557630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.558279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.558564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.558837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.578701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:21.583549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:21.585597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:21.587534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 246/488] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:21.595219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.618432 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.695558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef3d69-0ee7-445a-94d6-2bc3e98fd49d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.778570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.779218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.779507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.779783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.780483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:21.780789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.782235 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.800689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6674f-c5fe-40a6-88fb-6c40b7028ba5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.815312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 243/489] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:21.820698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.852364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e9483-b603-4f9c-9c8a-a0f5e69974ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.853001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e9483-b603-4f9c-9c8a-a0f5e69974ea None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.853279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e9483-b603-4f9c-9c8a-a0f5e69974ea None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.853553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e9483-b603-4f9c-9c8a-a0f5e69974ea None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.854270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e9483-b603-4f9c-9c8a-a0f5e69974ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:21.854574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e9483-b603-4f9c-9c8a-a0f5e69974ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:21.855964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e9483-b603-4f9c-9c8a-a0f5e69974ea None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:21.868233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 247/490] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:21.870126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d2f19-67fa-45ab-9063-81d672bd7979 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.870412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d2f19-67fa-45ab-9063-81d672bd7979 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.870692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d2f19-67fa-45ab-9063-81d672bd7979 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:21.876332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 248/491] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:21.877084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d2f19-67fa-45ab-9063-81d672bd7979 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:21.899821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.953051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1453a02-1bc3-43e9-acbd-63004a4270a9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:21.979512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:21.992194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61238a77-b6f4-40bc-85ea-5fe744e3f730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:21.996828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61238a77-b6f4-40bc-85ea-5fe744e3f730 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:21.996828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61238a77-b6f4-40bc-85ea-5fe744e3f730 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:21.996828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61238a77-b6f4-40bc-85ea-5fe744e3f730 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.002188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61238a77-b6f4-40bc-85ea-5fe744e3f730 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.004018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61238a77-b6f4-40bc-85ea-5fe744e3f730 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.016133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d2f19-67fa-45ab-9063-81d672bd7979 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-61238a77-b6f4-40bc-85ea-5fe744e3f730 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:22.019849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:22.020768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:22.021647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:22.028998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 244/492] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:22.030575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afc8f0-b8dc-4cd2-9831-230b66644a61 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.031155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afc8f0-b8dc-4cd2-9831-230b66644a61 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.031619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afc8f0-b8dc-4cd2-9831-230b66644a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.052576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-58afc8f0-b8dc-4cd2-9831-230b66644a61 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:22.053474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.054124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.054435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-58afc8f0-b8dc-4cd2-9831-230b66644a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:22.054548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.054714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.067745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.069472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:22.089194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:22.090246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:22.091178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:22.091178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:22.091178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:22.091178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:22.091178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 249/493] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:22.091178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.152032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 245/494] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:22.155755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afc8f0-b8dc-4cd2-9831-230b66644a61 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.188283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.238504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8ee9f-2b47-4861-8e8e-98370097334c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.275728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.276455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.276746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.277021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.277816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:22.278186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.279698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.283400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afc8f0-b8dc-4cd2-9831-230b66644a61 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.297384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.321085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62378fb-1fbb-4a3e-968e-a4fc30e5d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.321704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62378fb-1fbb-4a3e-968e-a4fc30e5d861 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.321985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62378fb-1fbb-4a3e-968e-a4fc30e5d861 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.322262 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62378fb-1fbb-4a3e-968e-a4fc30e5d861 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.323233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62378fb-1fbb-4a3e-968e-a4fc30e5d861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:22.323544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62378fb-1fbb-4a3e-968e-a4fc30e5d861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.324936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62378fb-1fbb-4a3e-968e-a4fc30e5d861 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.325471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 250/495] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:22.326373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.337536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 246/496] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:22.338959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfc1f8-4bb7-4f43-a0d4-e0a4793a145b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.339474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfc1f8-4bb7-4f43-a0d4-e0a4793a145b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.339875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfc1f8-4bb7-4f43-a0d4-e0a4793a145b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.341604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 247/497] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:22.342470 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfc1f8-4bb7-4f43-a0d4-e0a4793a145b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.431510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ac4c8-9aea-413d-83c0-7e45b9f40d66 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.449828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfc1f8-4bb7-4f43-a0d4-e0a4793a145b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.454138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.465452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.466099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.466386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.466667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.476682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:22.481589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:22.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:22.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:22.484512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a03dd-b30f-45a9-989e-8feb30711fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.484608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 251/498] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:22.484677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a03dd-b30f-45a9-989e-8feb30711fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.484677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a03dd-b30f-45a9-989e-8feb30711fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.484811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.484878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a03dd-b30f-45a9-989e-8feb30711fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.534788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.561840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a03dd-b30f-45a9-989e-8feb30711fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.563327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a03dd-b30f-45a9-989e-8feb30711fa1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.588234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cea047-ae85-415a-b3d3-d9e97afb25ac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.622500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b121-7052-44c9-a479-76d2d1ac1b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.623159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b121-7052-44c9-a479-76d2d1ac1b64 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.623539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b121-7052-44c9-a479-76d2d1ac1b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.623817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b121-7052-44c9-a479-76d2d1ac1b64 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.630934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b121-7052-44c9-a479-76d2d1ac1b64 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.632455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b121-7052-44c9-a479-76d2d1ac1b64 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cac3b121-7052-44c9-a479-76d2d1ac1b64 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:22.645883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:22.646832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:22.647756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 252/499] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:22.648565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b5e76-23c6-4d29-9d8f-ba0f36332560 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.648565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b5e76-23c6-4d29-9d8f-ba0f36332560 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.648779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b5e76-23c6-4d29-9d8f-ba0f36332560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.666342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d95b5e76-23c6-4d29-9d8f-ba0f36332560 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:22.667571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d95b5e76-23c6-4d29-9d8f-ba0f36332560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:22.669597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 248/500] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:22.670914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc29c2-ba2f-47f5-8572-61d68e88494f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.671281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc29c2-ba2f-47f5-8572-61d68e88494f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.671568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc29c2-ba2f-47f5-8572-61d68e88494f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.672877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 249/501] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:22.673641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc29c2-ba2f-47f5-8572-61d68e88494f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.707585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.738428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 253/502] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:22.739156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b5e76-23c6-4d29-9d8f-ba0f36332560 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.752137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.785878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc29c2-ba2f-47f5-8572-61d68e88494f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.821990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.822615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.822905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.823210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.823932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:22.824242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.825706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.833746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 250/503] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:22.834543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.848713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b5e76-23c6-4d29-9d8f-ba0f36332560 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.883333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4e0fb-6732-41a9-835b-c3498f858965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.883958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4e0fb-6732-41a9-835b-c3498f858965 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.884241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4e0fb-6732-41a9-835b-c3498f858965 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.884524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4e0fb-6732-41a9-835b-c3498f858965 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.885316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4e0fb-6732-41a9-835b-c3498f858965 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:22.885628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4e0fb-6732-41a9-835b-c3498f858965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.887100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4e0fb-6732-41a9-835b-c3498f858965 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.904921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 254/504] 10.222.0.49 () {56 vars in 1137 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:22.905934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead5fd-f097-4ebe-9352-cbfcb1cd547e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.906222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead5fd-f097-4ebe-9352-cbfcb1cd547e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.906502 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead5fd-f097-4ebe-9352-cbfcb1cd547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.923913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d6ead5fd-f097-4ebe-9352-cbfcb1cd547e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:22.925138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d6ead5fd-f097-4ebe-9352-cbfcb1cd547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:22.935090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f563f5-81f8-43c7-808f-fc59ac84e8c4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:22.948681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.969275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:22.970168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:22.975106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:22.975106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:22.975106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:22.983038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:22.984588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:22.990443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 255/505] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:22.991545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ead5fd-f097-4ebe-9352-cbfcb1cd547e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:22.999970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:23.000807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:23.001648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:23.001648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:23.001648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:23.001648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:23.001648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 251/506] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:23.002280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.097037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ead5fd-f097-4ebe-9352-cbfcb1cd547e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.105155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b9ba6-7a89-4036-a9e1-235ab2c86ba6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.132483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.133117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.133397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.133667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.139671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.140336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.140647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.140953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.143947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:23.149346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:23.150257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 256/507] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:23.151481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.154795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.156639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:23.171978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:23.172855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:23.173772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:23.173772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:23.173772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:23.173772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:23.173979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.174055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 252/508] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:23.174055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.216572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.253000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027623-9a40-4822-97e8-eeea7724b8df None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.279045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef07c0d-0858-4e69-9cb1-44a9c281b27e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.289657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.290589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.290879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.291160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.314869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.315572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.315857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.316179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.316877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:23.317237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.318748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.332016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 253/509] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:23.332821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.363432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.364899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.386989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.444320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.449010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d8e4f-f561-4853-8a8d-5c55e3927aac None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.479252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 257/510] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:23.480300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.488051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.488841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.489199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.489539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.501371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.502844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:23.520286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:23.521374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:23.522515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:23.522515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:23.522515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:23.522515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:23.522515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 254/511] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:23.522515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.607295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befa797-1f6e-452e-b113-2de12fa7c0ef None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.628178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.628583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45f0d1-5eab-43e4-8175-5b8845fcd774 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.642830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.643472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.643765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.644088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.644802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:23.645111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.646547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.655889 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 258/512] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:23.656604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.667220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.667866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.668146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.668425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.669127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:23.669437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.670833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.682937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 255/513] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:23.683825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.697497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.760911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ad103-dc25-4273-ae25-5ba42e4c5f25 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.797684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.798280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b929a1-7a71-4429-a804-60dbaa805b9f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.798419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.798610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.798891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.809145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.810751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:23.824241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:23.825197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:23.826075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 259/514] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:23.826532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.842899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.843581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.843865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.844141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.854749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.856260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:23.871519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:23.872364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:23.873196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:23.873196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:23.873196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:23.873196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:23.873401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.873468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 256/515] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:23.873468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:23.930324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cf9d8-7433-4122-885d-8d042fa76fea None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.935946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:23.964788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:23.965403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:23.965690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:23.965977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:23.966665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:23.966972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:23.968407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:23.974771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7547e-29dd-4cc2-95de-994e5a106411 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:23.980603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 260/516] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:23.981401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.008916 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.009553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.009840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.010113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.020699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.022143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:24.036745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:24.037623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:24.043545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:24.043545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:24.043545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:24.043545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:24.043545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 257/517] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:24.043545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.081997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a491ee3-22e9-499e-b8a4-bf0c73eed4b4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.083583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.117740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.118438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.118729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.119008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.127086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.128694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:24.140999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:24.141951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 261/518] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:24.144255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.153368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da450a49-56a6-4510-99f6-fcdcbac92044 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.167812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.188724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.189333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.189620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.189898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.190707 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:24.191080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.192698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.212490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 258/519] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:24.213498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.244007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2a526-d644-4653-904a-6f2b24e953cf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.278500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.279116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.279435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.279708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.280390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:24.280698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.282055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.293939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 262/520] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:24.294716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.317115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a7ba-d469-4d3b-bf8c-73aeadc1a2d0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.318875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.354198 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.354824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.355111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.355423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.356104 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.357554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.393206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa1fd-786e-421c-93fb-eb4f12e741b3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.427289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.440002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.440002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.440002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.440002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:24.440002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.440002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.442956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 263/521] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:24.454694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 259/522] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:24 2022] POST /identity/v3/users => generated 247 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:24.454914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.455002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.482931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.552022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.573456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fccd97-67aa-4cdf-b45a-e74a39f9afdc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.573812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3947ba9-4661-4eaf-9aa9-d3c131322518 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.611462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.612091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.612385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.612665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.613465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=9072775865604c73ae8361a047945464) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:24.613782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.615259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.622034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 260/523] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/role_assignments?user.id=9072775865604c73ae8361a047945464&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:24.622858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.623561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80270113-dca9-455e-97fa-61479effeeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.624396 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80270113-dca9-455e-97fa-61479effeeec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.624566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80270113-dca9-455e-97fa-61479effeeec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.624861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80270113-dca9-455e-97fa-61479effeeec None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.625734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80270113-dca9-455e-97fa-61479effeeec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:24.626095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80270113-dca9-455e-97fa-61479effeeec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.627682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80270113-dca9-455e-97fa-61479effeeec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.652415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 264/524] 10.222.0.49 () {56 vars in 1137 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:24.653136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80270113-dca9-455e-97fa-61479effeeec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.765811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.771177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57747ecc-2525-46f1-850c-bf0db2c717ea None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.786225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80270113-dca9-455e-97fa-61479effeeec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.825399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.826270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.826565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.826912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.830139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:24.831143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:24.831749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:24.832242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:24.834786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.837617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.841699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:24.843550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:24.853059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:24.854164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:24.855291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:24.856351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 261/525] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:24.857338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.858131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:24.861554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:24.862562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 265/526] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:24.870201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:24.986349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cf6b2-1a61-47ca-b5b7-e814ef569b22 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:24.998325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057bc16-5993-4329-bc00-3659a015c7a8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.036401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.037022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.037368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.037681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.038285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.038520 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:25.038870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.039560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.040057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.040467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.040705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.063159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 266/527] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:25.064107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:25.074140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.122715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.124623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.131580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.185211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ce56-3d42-4c9a-ad7e-32d70a3e2428 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.231388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.232407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.232700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.232982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.235683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:25.236145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.238322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.246347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 262/528] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:25.247637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:25.260558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 267/529] 10.222.0.49 () {56 vars in 1139 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:25.261859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:25.336196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.370694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbb4e8-a579-46ab-9db0-733f6fbb48f7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.383894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952ae82-8aa2-44f8-9ebb-a9b0f29735a4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.397903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.422828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.423841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.432554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.432554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.432735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.432735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.432735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.432735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.433024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=2dbb92bcc5054287a96fbfac03d2975a) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:25.433024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.433024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.436410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.438039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.443661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 263/530] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/role_assignments?user.id=2dbb92bcc5054287a96fbfac03d2975a&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:25.444721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:25.459460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:25.460441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:25.462228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:25.462228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:25.462228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:25.462228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:25.462228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 268/531] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:25.469243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:25.655153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be63c2-7b94-4c08-9da3-525037c81d96 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.681897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152f579-c74b-4e75-abb6-2a99070617f0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.700693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.700693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.700693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.705439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.705439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:25.705439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.705439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.710587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.732880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 269/532] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:25.740261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:25.742996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:25.743867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:25.744329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:25.744749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:25.807565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:25.874624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:25.876166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:25.962350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2ffd0-255a-4121-bf94-f42d06047ba8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:25.981217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 264/533] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:25.986552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.008519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.009464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.010016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.010485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.017831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.019734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.020766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:26.033071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:26.040263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:26.041085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 270/534] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:26.043179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.137635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.147577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b603d1e-105b-4ff2-acc6-f02da3f63019 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.150179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24966-f3bd-479a-a719-ec9a3cff755b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.187058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.188090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.188571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.189058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.189997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:26.190499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.192175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.204868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.205612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.205820 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.206195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.206418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 271/535] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:26.213003 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.232355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.233866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:26.264717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:26.265717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:26.266541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:26.266541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:26.266541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:26.266541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:26.273135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 265/536] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:26.274107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.322009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b7a1a-3449-44f8-9483-0009a5bc0144 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.363083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.364121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.364598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.365065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.366044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=700720d6f66e4de2a65b808afce70e39) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:26.366832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.371078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.381258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 272/537] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/role_assignments?user.id=700720d6f66e4de2a65b808afce70e39&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:26.427595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.464850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e68005-50c9-466f-8b22-34a7518ca72e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.501388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.502052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.502335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.502612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.503378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:26.503683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.505083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.519131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 266/538] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:26.524556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.549791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.648858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54d755-d363-45af-9cd1-d080490b2f87 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.686818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520cdd0-d9be-491e-bb1a-2f5e0579fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.687613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520cdd0-d9be-491e-bb1a-2f5e0579fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.687849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520cdd0-d9be-491e-bb1a-2f5e0579fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.688130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520cdd0-d9be-491e-bb1a-2f5e0579fbc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.695796 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520cdd0-d9be-491e-bb1a-2f5e0579fbc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.697282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520cdd0-d9be-491e-bb1a-2f5e0579fbc4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-8520cdd0-d9be-491e-bb1a-2f5e0579fbc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:26.709932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:26.710867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:26.711724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 273/539] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:26.720634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.725669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e436-8ba8-445d-bb0c-36337b7e236f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.726282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e436-8ba8-445d-bb0c-36337b7e236f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.727334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e436-8ba8-445d-bb0c-36337b7e236f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.729326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 274/540] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:26.735442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0236-b3a0-4eb5-bc0d-08c084e1091e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.736024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0236-b3a0-4eb5-bc0d-08c084e1091e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.736535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0236-b3a0-4eb5-bc0d-08c084e1091e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.759533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3bcf0236-b3a0-4eb5-bc0d-08c084e1091e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:26.761235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3bcf0236-b3a0-4eb5-bc0d-08c084e1091e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:26.842070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 275/541] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:26.876137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875e0c-6e82-4e79-ac6b-870481ebba50 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.916346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.921584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf0236-b3a0-4eb5-bc0d-08c084e1091e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.923639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b65afb-20ff-4236-8a7d-12aa7e3d0a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.924699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65afb-20ff-4236-8a7d-12aa7e3d0a4c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.925117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65afb-20ff-4236-8a7d-12aa7e3d0a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.925516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65afb-20ff-4236-8a7d-12aa7e3d0a4c None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.926413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b65afb-20ff-4236-8a7d-12aa7e3d0a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:26.927066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b65afb-20ff-4236-8a7d-12aa7e3d0a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.929051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.931290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b65afb-20ff-4236-8a7d-12aa7e3d0a4c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.947070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 267/542] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:26.957912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae7f3-0d95-4d83-a3a3-64d4ecc23825 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.957912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae7f3-0d95-4d83-a3a3-64d4ecc23825 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.957912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae7f3-0d95-4d83-a3a3-64d4ecc23825 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.961636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 268/543] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:26.962783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae7f3-0d95-4d83-a3a3-64d4ecc23825 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.065755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf0236-b3a0-4eb5-bc0d-08c084e1091e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.097715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae7f3-0d95-4d83-a3a3-64d4ecc23825 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.111543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e8438-89ef-41ac-9a95-183fe9d3c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.112209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e8438-89ef-41ac-9a95-183fe9d3c073 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.112495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e8438-89ef-41ac-9a95-183fe9d3c073 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.112775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e8438-89ef-41ac-9a95-183fe9d3c073 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.125870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e8438-89ef-41ac-9a95-183fe9d3c073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-300e8438-89ef-41ac-9a95-183fe9d3c073 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:27.130879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:27.131766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:27.132547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 276/544] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:27.139225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feafe2a-93b3-4a03-b392-773883a360b2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.139225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feafe2a-93b3-4a03-b392-773883a360b2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.139225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feafe2a-93b3-4a03-b392-773883a360b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.154851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.155845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5feafe2a-93b3-4a03-b392-773883a360b2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:27.156598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.157044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5feafe2a-93b3-4a03-b392-773883a360b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:27.157390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.157861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.158857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=09cabb184e704aecbb0a3bfb1cd0a584) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:27.159567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.162132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.170486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 269/545] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/role_assignments?user.id=09cabb184e704aecbb0a3bfb1cd0a584&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:27.171645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.241244 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.310456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8defd79-eee3-49fd-aeda-a53ca548cd21 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.325389 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.342984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 277/546] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:27.378552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.379242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.379566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.379872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.380626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:27.380959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.382392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.399538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 270/547] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:27.401476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feafe2a-93b3-4a03-b392-773883a360b2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.420212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.569421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feafe2a-93b3-4a03-b392-773883a360b2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.626128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676260-5805-4296-86fa-cf38118f3df6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.644130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.652616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.653281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.653568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.653847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.666069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.666862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.667169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.667479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.680516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:27.680789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.682287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:27.686261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:27.687117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:27.687940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:27.689886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 271/548] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:27.694693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:27.714413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:27.715829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:27.716687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:27.716687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:27.716687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:27.716687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:27.716687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 278/549] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:27.723397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.833486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.837368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202773d-7ed7-4e04-91e0-83f2d443dc8c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.880933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.881539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.881857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.882212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.883629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:27.886007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.887439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.908206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 272/550] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:27.915391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.936776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cae05-9711-43eb-a4c4-ab384eaf9a07 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.971886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.973434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50048e1-bad9-40af-adcb-8628c2223682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.974062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50048e1-bad9-40af-adcb-8628c2223682 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.974347 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50048e1-bad9-40af-adcb-8628c2223682 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.974623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50048e1-bad9-40af-adcb-8628c2223682 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.975353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50048e1-bad9-40af-adcb-8628c2223682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:27.975658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50048e1-bad9-40af-adcb-8628c2223682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.977609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50048e1-bad9-40af-adcb-8628c2223682 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.990388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 279/551] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:27.992014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4a51-2636-4998-bbdb-df86a406f544 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.997760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4a51-2636-4998-bbdb-df86a406f544 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.997760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4a51-2636-4998-bbdb-df86a406f544 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.997760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 280/552] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:27.998660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4a51-2636-4998-bbdb-df86a406f544 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.055573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f5a26-465f-44cc-8b41-3f4b11ab5333 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.107723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f2d42-e918-4020-b198-083ac5d3d221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.108708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2d42-e918-4020-b198-083ac5d3d221 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.109000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2d42-e918-4020-b198-083ac5d3d221 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.109285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2d42-e918-4020-b198-083ac5d3d221 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.114153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4a51-2636-4998-bbdb-df86a406f544 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.122560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.124462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f2d42-e918-4020-b198-083ac5d3d221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.126363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f2d42-e918-4020-b198-083ac5d3d221 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d13f2d42-e918-4020-b198-083ac5d3d221 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:28.146852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:28.149201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 273/553] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ac720-3ccd-486a-961f-b5bed7efcf69 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ac720-3ccd-486a-961f-b5bed7efcf69 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.151033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ac720-3ccd-486a-961f-b5bed7efcf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.159210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.159210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.159210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.159210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.165600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.167634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.172581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e18ac720-3ccd-486a-961f-b5bed7efcf69 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:28.173896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e18ac720-3ccd-486a-961f-b5bed7efcf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:28.181371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:28.182924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:28.184041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 281/554] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:28.184737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.255482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 274/555] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:28.256574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ac720-3ccd-486a-961f-b5bed7efcf69 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.291342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.309674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befeb03-0fa9-41a1-81dd-1903c7f2e06b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.348337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.348994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.349280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.349839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.350358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:28.350667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.352732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.366410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 282/556] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:28.367632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.397867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.400778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ac720-3ccd-486a-961f-b5bed7efcf69 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.443301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.444270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.444558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.444838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.445959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:28.446277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.448402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.492492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8663b-add4-432d-a104-a494ab5b046e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.500672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 275/557] 10.222.0.49 () {56 vars in 1135 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:28.502075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.536100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.536979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.537373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.537821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.550800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:28.556308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:28.557856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:28.558864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:28.560753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 283/558] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:28.563234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.623743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.651812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf175a05-3454-41ad-b4a5-6be29c2c2cea None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.660515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.678857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cede047-d6bb-4232-aef4-8adc52e586ab None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.703095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.703095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.703095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.703095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.712665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.714544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.723353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.724170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.724545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.724914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:28.729772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:28.730766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:28.731739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:28.734436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 276/559] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:28.735427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.809731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.811453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.848479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e7865-981f-46eb-a66c-704ee2f41aac None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.893239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.894780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.895284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.895863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.896832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:28.897332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.899667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.905747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.909851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 277/560] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:28.910986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:28.914601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.938767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 284/561] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:28.938767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.017429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fda4-ad09-4447-8a8c-3a5f271285d7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.051179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8070f4-1e50-4c3a-994c-08205d42ae56 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.056632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5baac5a-a224-4cad-bc8d-ff2d8aca5973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.057509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baac5a-a224-4cad-bc8d-ff2d8aca5973 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.057963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baac5a-a224-4cad-bc8d-ff2d8aca5973 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.058384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baac5a-a224-4cad-bc8d-ff2d8aca5973 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.059521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baac5a-a224-4cad-bc8d-ff2d8aca5973 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:29.060549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baac5a-a224-4cad-bc8d-ff2d8aca5973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.062608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baac5a-a224-4cad-bc8d-ff2d8aca5973 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.085528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 278/562] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:29.091390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7858a6-4224-43ec-bc69-a2c9c5ee0d93 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.094799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.094951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7858a6-4224-43ec-bc69-a2c9c5ee0d93 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.094951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7858a6-4224-43ec-bc69-a2c9c5ee0d93 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.095119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.095249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 279/563] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:29.095357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.095357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.095618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7858a6-4224-43ec-bc69-a2c9c5ee0d93 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.110484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.111962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:29.129255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:29.130445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:29.131429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:29.131429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:29.131429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:29.131429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:29.131429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 285/564] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:29.131711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.173607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.183969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.226986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7858a6-4224-43ec-bc69-a2c9c5ee0d93 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.250688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511e454-0d0d-4cf1-9720-4c8bf3b90489 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.269299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbfcce-383f-4312-bbfd-46d026c8fb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.270398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbfcce-383f-4312-bbfd-46d026c8fb41 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.270696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbfcce-383f-4312-bbfd-46d026c8fb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.271024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbfcce-383f-4312-bbfd-46d026c8fb41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.281982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbfcce-383f-4312-bbfd-46d026c8fb41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.283547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbfcce-383f-4312-bbfd-46d026c8fb41 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.288595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.289247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.289540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.289817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-95cbfcce-383f-4312-bbfd-46d026c8fb41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:29.299572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:29.300597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:29.301611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:29.301611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:29.301611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:29.301611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:29.301611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 280/565] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:29.302537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46250f-bd97-4f61-acc2-b75f5e9645f2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.302537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46250f-bd97-4f61-acc2-b75f5e9645f2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.302697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46250f-bd97-4f61-acc2-b75f5e9645f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.306849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.308371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.323849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ba46250f-bd97-4f61-acc2-b75f5e9645f2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:29.325095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ba46250f-bd97-4f61-acc2-b75f5e9645f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:29.326521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:29.327443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:29.328330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:29.328330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:29.328330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:29.328330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:29.328330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 286/566] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:29.329264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.405246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 281/567] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:29.406370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46250f-bd97-4f61-acc2-b75f5e9645f2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.430155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.440239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f7400-e93d-4e04-b766-55667189884b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.450118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.472668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.473461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.473800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.474292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.474993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:29.475793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.478189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.490216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 287/568] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:29.491070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.512739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46250f-bd97-4f61-acc2-b75f5e9645f2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.549510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef9f26-255b-486a-ae61-6455b888cbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.550139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef9f26-255b-486a-ae61-6455b888cbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.550422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef9f26-255b-486a-ae61-6455b888cbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.550698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef9f26-255b-486a-ae61-6455b888cbc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.551442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef9f26-255b-486a-ae61-6455b888cbc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:29.551749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef9f26-255b-486a-ae61-6455b888cbc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.553134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef9f26-255b-486a-ae61-6455b888cbc6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.565286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 282/569] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:29.566542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8b00-7a8e-45db-b0e3-a637e228aac7 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.566885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8b00-7a8e-45db-b0e3-a637e228aac7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.567169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8b00-7a8e-45db-b0e3-a637e228aac7 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.568408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 283/570] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:29.569127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8b00-7a8e-45db-b0e3-a637e228aac7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.599502 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc79a5-c0d8-4ff5-a027-9141acdb95d9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.633557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.634173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.634453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.634727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.635431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:29.635738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.637101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.648274 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 288/571] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:29.649163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.663802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.673742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.677653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8b00-7a8e-45db-b0e3-a637e228aac7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.713021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.713699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.713992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.714269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.725326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:29.731282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:29.732394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:29.733415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:29.735576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 284/572] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:29.736622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.754082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc859323-a8f1-44d8-820b-83fc7ef978a1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.788627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff6dc0-582a-496a-98c4-506182d07ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.789339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff6dc0-582a-496a-98c4-506182d07ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.789631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff6dc0-582a-496a-98c4-506182d07ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.789907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff6dc0-582a-496a-98c4-506182d07ab0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.797815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff6dc0-582a-496a-98c4-506182d07ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.799318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff6dc0-582a-496a-98c4-506182d07ab0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2fff6dc0-582a-496a-98c4-506182d07ab0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:29.812265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:29.813241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:29.814186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 289/573] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:29.816564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed1c48-1f4a-40aa-a9cf-73279689265c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.816857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed1c48-1f4a-40aa-a9cf-73279689265c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.817138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed1c48-1f4a-40aa-a9cf-73279689265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.838997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b6ed1c48-1f4a-40aa-a9cf-73279689265c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:29.840240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b6ed1c48-1f4a-40aa-a9cf-73279689265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:29.843938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d7309-61f1-423d-a3ab-e142aa1b1529 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:29.881086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:29.881787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:29.882069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:29.882343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:29.889601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:29.891060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:29.902251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:29.903641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:29.904516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:29.904927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:29.904999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 285/574] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:29.904999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.918422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 290/575] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:29.918422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed1c48-1f4a-40aa-a9cf-73279689265c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:29.926695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.004209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a396b6a-8477-4eb0-b79a-94c1b0cf7b41 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.018165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed1c48-1f4a-40aa-a9cf-73279689265c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.037610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.038369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.038864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.039270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.046389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.047901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.052604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.053221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.053503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.053777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.054489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.055317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.056689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:30.060154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:30.061019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:30.061895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 286/576] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:30.062374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.071110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 291/577] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.072364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.135592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.144749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.165504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856b8cb-caa8-4ba3-9e95-6104d5ce9c55 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.174204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f289954-591c-4562-b523-805a6a96c16d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.202343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.202996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.203339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.203649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.204558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.204917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.206443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.208789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.210147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.210437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.210721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.212097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.212414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.213841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.225684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 287/578] 10.222.0.49 () {56 vars in 1137 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.226586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.226954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 292/579] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.227877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.326653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee9db0-6683-462d-81b3-9d2a4f3beb5d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.338432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71c176-ee19-44bb-b5c1-57114a200ba9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.361223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.361862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.362001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.362171 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.362424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.369560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.373437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.373577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:30.374052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.374330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.374603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.375375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.375687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.377099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:30.378383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:30.379683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 288/580] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:30.387263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.389421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 293/581] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.390230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.486742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505c36a-303e-4628-b551-bcda6971a55a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.489402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d3392-c386-4493-a031-79da39b7568e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.526410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.527114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.527450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.527759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.531105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.531781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.532058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.532332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.539025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.541094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.543887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.545369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:30.555017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:30.556240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:30.557353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:30.557353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:30.557353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:30.557353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:30.557353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 294/582] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:30.557353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:30.559805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:30.560737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:30.568328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:30.568328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:30.568328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:30.568328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:30.568328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 289/583] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:30.568328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.593284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.610794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.659346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cda477-1b20-4977-b42a-76c95a9ea0cc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.666446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bb74c-75bc-4122-b26b-348bd5e1bc87 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.692281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.692903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.693185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.693533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.694215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.694514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.695925 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.701035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.701656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.701952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.702215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.702998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=66f7db7eea94472c82cda44c569d9550) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.703350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.703944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 290/584] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.704765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.704876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.715593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 295/585] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/role_assignments?user.id=66f7db7eea94472c82cda44c569d9550&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.716433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.857860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.860432 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c37ed6-1df4-4fee-9efd-c5a8131aa576 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.861781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b280-4808-4694-af89-2d9e670fcbd2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:30.876047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:30.901127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.901878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.902229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.902577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/9072775865604c73ae8361a047945464/roles/b0563004ff664008b6db05941cf9c9b9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.904397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:30.905028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:30.905316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:30.905783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:30.906591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:30.906907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.908354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.920888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=9072775865604c73ae8361a047945464, role_id=b0563004ff664008b6db05941cf9c9b9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:30.921626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 296/586] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:30.922351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:30.927270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:30.999790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 291/587] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:30 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/9072775865604c73ae8361a047945464/roles/b0563004ff664008b6db05941cf9c9b9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:31.000763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.053876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577356ed-4a7d-4e75-b6b2-1c7c44b236f3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.105421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.106482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.106969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.107663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.108410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:31.108669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.110249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.124741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 297/588] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:31.125236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f063d18-249b-4b7f-a49f-80c5f344544e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.125662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.148047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.176103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.176779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.177054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.177340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.192143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.194141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:31.215389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:31.216332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:31.217216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:31.217216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:31.217216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:31.217216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:31.217216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 292/589] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:31.217493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.234372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.259655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3467fef-4498-4edc-a607-cac58c58be1f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.306528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.308308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.308806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.309262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.320389 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.322859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:31.339534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:31.340545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:31.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:31.344634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 298/590] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:31.345576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1f0ae-8b13-46f2-b642-d3b9c5ad798e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.345882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.399977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.400698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.400984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.401264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.413075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.415068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:31.430883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:31.431901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:31.432826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 293/591] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:31.433983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.437335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.460382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04768ff8-bfdc-49d2-9823-51731a84f02b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.505581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.506500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.507138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.511091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.512773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:31.513357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.516924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.539276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 299/592] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:31.540409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.541444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.567834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea8c1-3ed0-43ed-bc3e-276262c99320 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.606239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.607488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.607984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.608815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.610085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:31.610780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.613013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.629955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 294/593] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:31.631069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.681935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f9f4a-c2a6-499a-bae5-2272833f7092 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.727413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.728040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.728321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.728657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.729450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:31.729793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.731235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.742118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.760620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 300/594] 10.222.0.49 () {56 vars in 1137 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:31.760620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.776519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f46c3f-5539-447e-baf5-aad479dd6fed None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.813507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.814202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.814484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.814822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.816984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:31.824014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.825586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:31.838039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:31.839080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:01:31.840118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:31.844163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 295/595] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:31.845155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:31.890134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c945ba1-e92f-4cbf-a2fb-35f274ed9ed7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:31.937426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:31.937915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:31.938234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:31.938513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/2dbb92bcc5054287a96fbfac03d2975a/roles/b0563004ff664008b6db05941cf9c9b9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:31.955055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=2dbb92bcc5054287a96fbfac03d2975a, role_id=b0563004ff664008b6db05941cf9c9b9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:31.956707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:31.979590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b2bc0-0ca3-43be-b783-a0548e10fb44 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.015175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.015862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.016149 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.016430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.031421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 301/596] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:31 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/2dbb92bcc5054287a96fbfac03d2975a/roles/b0563004ff664008b6db05941cf9c9b9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:32.031421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.039962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.041652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:32.071924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:32.072930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:32.073789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:32.073789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:32.073789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:32.073789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:32.083771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 296/597] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:32.085871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.135146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.169096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.251034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6eeb0-24b6-4f76-9297-750572c6c5f2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.280875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e80ed-18d4-4c14-baf1-04b7a94e2b77 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.311243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.312276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.312764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.313240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.314263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:32.314860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.316629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.328778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.329455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.329759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.330005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.330706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:32.331009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.332426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.339784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 302/598] 10.222.0.49 () {56 vars in 1139 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:32.345134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.349790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 297/599] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:32.356061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.451932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70f5af-0f36-49da-8aae-43ce21a2859a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.454397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.465467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817e00b-486f-4c78-9acd-415825135bea None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.473581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.493619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.494966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.495501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.495942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.502536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.503270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.503697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.504221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.513536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.516081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.516345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.517940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:32.533525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:32.534542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:32.535645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:32.537403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 298/600] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:32.539587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:32.540464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:32.541326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:32.541326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:32.541326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:32.541326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:32.543035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 303/601] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:32.547284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.549556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.663521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60559878-b3e7-4f28-9cb9-b93c1b735beb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.686554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bfa0d-a2f2-455b-aed1-99be2c661549 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.701408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.702198 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.702646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.702948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.703880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:32.704292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.705829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.718746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 299/602] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:32.724877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.726543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.727909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.728356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.728771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.729602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:32.730050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.731577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.745628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 304/603] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:32.751518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:32.760814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.771727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:32.839314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d890f-7e5a-472a-ac8e-2ad429b129d2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.859058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21606b9-e457-4033-aaad-01066a15f180 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:32.877314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545ed83-d19a-4324-9d55-7236215f8355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.878354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545ed83-d19a-4324-9d55-7236215f8355 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.878511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545ed83-d19a-4324-9d55-7236215f8355 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.878893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545ed83-d19a-4324-9d55-7236215f8355 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/700720d6f66e4de2a65b808afce70e39/roles/b0563004ff664008b6db05941cf9c9b9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.898140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545ed83-d19a-4324-9d55-7236215f8355 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=700720d6f66e4de2a65b808afce70e39, role_id=b0563004ff664008b6db05941cf9c9b9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.900358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545ed83-d19a-4324-9d55-7236215f8355 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.905385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ce8af-3aae-4be7-8f65-465c633fe8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:32.906613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce8af-3aae-4be7-8f65-465c633fe8da None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.907072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce8af-3aae-4be7-8f65-465c633fe8da None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.907528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce8af-3aae-4be7-8f65-465c633fe8da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.916011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ce8af-3aae-4be7-8f65-465c633fe8da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:32.917560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ce8af-3aae-4be7-8f65-465c633fe8da None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-5c2ce8af-3aae-4be7-8f65-465c633fe8da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:32.931019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:32.931932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:32.932848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 305/604] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:32.938528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafa2cb-a1f5-4cd1-93f7-801c564a7616 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:32.938528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafa2cb-a1f5-4cd1-93f7-801c564a7616 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:32.938528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafa2cb-a1f5-4cd1-93f7-801c564a7616 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:32.938528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 306/605] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:32.938840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafa2cb-a1f5-4cd1-93f7-801c564a7616 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.026547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.027358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 300/606] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:32 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/700720d6f66e4de2a65b808afce70e39/roles/b0563004ff664008b6db05941cf9c9b9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:33.028915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28af8-0674-46f8-830f-3dcdf0d7c94d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.029215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28af8-0674-46f8-830f-3dcdf0d7c94d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.029496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28af8-0674-46f8-830f-3dcdf0d7c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.056936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafa2cb-a1f5-4cd1-93f7-801c564a7616 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.072101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-94a28af8-0674-46f8-830f-3dcdf0d7c94d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:33.073342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-94a28af8-0674-46f8-830f-3dcdf0d7c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:33.100169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.100919 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.101204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.101478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.102277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:33.102583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.104129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.120923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 307/607] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:33.129064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.201038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 301/608] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:33.245942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a28af8-0674-46f8-830f-3dcdf0d7c94d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.254886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.301199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef96b8-3d09-446a-bb2f-7b9e153d7d96 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.343067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.344043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.344505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.344979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/09cabb184e704aecbb0a3bfb1cd0a584/roles/b0563004ff664008b6db05941cf9c9b9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.348581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.361324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=09cabb184e704aecbb0a3bfb1cd0a584, role_id=b0563004ff664008b6db05941cf9c9b9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.363010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.376891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a28af8-0674-46f8-830f-3dcdf0d7c94d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.414794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425b290-bd47-446e-ba05-8a46855e6c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.415591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425b290-bd47-446e-ba05-8a46855e6c80 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.415902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425b290-bd47-446e-ba05-8a46855e6c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.416207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425b290-bd47-446e-ba05-8a46855e6c80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.427584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425b290-bd47-446e-ba05-8a46855e6c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6425b290-bd47-446e-ba05-8a46855e6c80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:33.433098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:33.439220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:33.441831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 302/609] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:33.442538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 308/610] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:33 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/09cabb184e704aecbb0a3bfb1cd0a584/roles/b0563004ff664008b6db05941cf9c9b9 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:33.442622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66907939-f68f-4736-94e1-241f1ca3c7d0 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.442622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66907939-f68f-4736-94e1-241f1ca3c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.442622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66907939-f68f-4736-94e1-241f1ca3c7d0 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.443583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 303/611] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:33.447223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.450371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019ebe6-8dc3-4053-bac6-29d24a8dada6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.450670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019ebe6-8dc3-4053-bac6-29d24a8dada6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.450953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019ebe6-8dc3-4053-bac6-29d24a8dada6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.487038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d019ebe6-8dc3-4053-bac6-29d24a8dada6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:33.489344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d019ebe6-8dc3-4053-bac6-29d24a8dada6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:33.622942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.637574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 304/612] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:33.663312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdad42-bd66-4604-acd8-c15a07865ae6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.681784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019ebe6-8dc3-4053-bac6-29d24a8dada6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.708795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.709497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.709901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.710292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.711711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:33.712139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.713999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.725664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 309/613] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:33.732020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.788730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.817929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019ebe6-8dc3-4053-bac6-29d24a8dada6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.845003 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14cc9d-cd3a-446d-8b71-926789c99adf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.858595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.859577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.860086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.860547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.873647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:33.880070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:33.881010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:33.881863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:33.884592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 305/614] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:33.895288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.897708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.904078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.904078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.904078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.928471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.930599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:33.957862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:33.958934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:33.960090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:33.960090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:33.960090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:33.960090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:33.965062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 310/615] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:33.968431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.025764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.106971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c252a-f770-4c5d-9ba6-a44778d36216 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.125180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.128737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f1c3-fb4b-40e8-98b8-a60988611eef None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.163437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.164078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.164953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.165255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.165988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:34.166817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.168260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.171649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.172372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.172660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.172995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.173696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:34.174006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.175385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.194055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 306/616] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:34.203011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 311/617] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:34.207410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.212956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.369234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd596-b7f8-4122-9d02-ee0d85968f8f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.390712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.413256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.415380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.416800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.417283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.441935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc17545-5647-4cb2-a9dd-311801fcecc1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.446369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.447449 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:34.478635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:34.480174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:34.481222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:34.481222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:34.481222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:34.481222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:34.483365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 307/618] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:34.489005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df8371-1f31-4662-9f25-db5884c7c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.490127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8371-1f31-4662-9f25-db5884c7c55e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.490688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8371-1f31-4662-9f25-db5884c7c55e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.491218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8371-1f31-4662-9f25-db5884c7c55e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.493665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.518984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df8371-1f31-4662-9f25-db5884c7c55e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.521069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df8371-1f31-4662-9f25-db5884c7c55e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-34df8371-1f31-4662-9f25-db5884c7c55e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:34.548543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:34.549624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:34.550855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 312/619] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:34.552721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61e528-29b4-4f11-bbaa-71693188be05 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.556295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61e528-29b4-4f11-bbaa-71693188be05 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.556295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61e528-29b4-4f11-bbaa-71693188be05 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.556295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 313/620] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:34.558716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61e528-29b4-4f11-bbaa-71693188be05 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.593923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.640533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b37845-f068-4188-a880-3d5f798d8c6c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.689026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7307a74-7a19-4f53-b082-2fc8bf76467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.689617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307a74-7a19-4f53-b082-2fc8bf76467c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.689935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307a74-7a19-4f53-b082-2fc8bf76467c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.690248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307a74-7a19-4f53-b082-2fc8bf76467c None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.691033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7307a74-7a19-4f53-b082-2fc8bf76467c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:34.691374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7307a74-7a19-4f53-b082-2fc8bf76467c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.693891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7307a74-7a19-4f53-b082-2fc8bf76467c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.705869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61e528-29b4-4f11-bbaa-71693188be05 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.711931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 308/621] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:34.713612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730ca-d757-4b68-a10e-230b40d8ee11 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.714518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730ca-d757-4b68-a10e-230b40d8ee11 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.715041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730ca-d757-4b68-a10e-230b40d8ee11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.739274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-412730ca-d757-4b68-a10e-230b40d8ee11 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:34.740828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-412730ca-d757-4b68-a10e-230b40d8ee11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:34.750904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.752097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.752455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.752837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:34.754175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:34.754537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:34.756497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:34.769626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.785271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 314/622] 10.222.0.49 () {56 vars in 1135 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:34.786335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.823337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 309/623] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:34.824543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412730ca-d757-4b68-a10e-230b40d8ee11 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:34.904013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:34.939291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cb354-4d75-4f74-9908-f462771ea9a8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.956894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412730ca-d757-4b68-a10e-230b40d8ee11 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:34.989024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:34.989720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:34.990008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:34.990287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.001464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.002941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.006755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.007824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.008415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.008894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:35.017341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:35.018308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:35.019276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 315/624] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:35.019858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.027373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.028959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:35.045591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:35.046533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:35.047498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:35.047498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:35.047498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:35.047498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:35.051595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 310/625] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:35.052496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.111566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.143813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6c559-3d27-4b0a-8526-ed2c67455d0a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.174005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c8694-c2e3-45a7-af7e-55857f50b6bb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.185872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.187238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.187810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.188262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.198734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.202933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:35.209104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:35.216537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 316/626] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:35.217568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.227343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.228165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.228971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.229402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.230626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:35.231084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.233334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.261451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 311/627] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:35.262865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.335746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404908d-34de-41ab-b626-975352f2ff9b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.352394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.374214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.374992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.375340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.375625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.376362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:35.376692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.378164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.388411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fd914-11af-4835-8f52-1656101d7282 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.394742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 317/628] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:35.394742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.430663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e5d19-d3e7-42f2-875a-b26c23c225f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.431605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e5d19-d3e7-42f2-875a-b26c23c225f2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.432039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e5d19-d3e7-42f2-875a-b26c23c225f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.432468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e5d19-d3e7-42f2-875a-b26c23c225f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.433324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e5d19-d3e7-42f2-875a-b26c23c225f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:35.433777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e5d19-d3e7-42f2-875a-b26c23c225f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.435333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e5d19-d3e7-42f2-875a-b26c23c225f2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.444253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 312/629] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:35.445771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bcc67-3aa1-4956-99b6-e039a2f9c917 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.446051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bcc67-3aa1-4956-99b6-e039a2f9c917 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.446327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bcc67-3aa1-4956-99b6-e039a2f9c917 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.451299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 313/630] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:35.451299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bcc67-3aa1-4956-99b6-e039a2f9c917 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.467171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.521779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9355ea-2fad-4a7f-91fd-92e28c85746c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.561306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81a023-7611-4178-bde6-7b52f08f34b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.562178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a023-7611-4178-bde6-7b52f08f34b7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.562290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a023-7611-4178-bde6-7b52f08f34b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.562540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a023-7611-4178-bde6-7b52f08f34b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.569728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bcc67-3aa1-4956-99b6-e039a2f9c917 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.602840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.608277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.613901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.613901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.613901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.616658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.618932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:35.631415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:35.632355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:35.633301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:35.635876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 314/631] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:35.637533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.644443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81a023-7611-4178-bde6-7b52f08f34b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.645958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81a023-7611-4178-bde6-7b52f08f34b7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.718370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.752054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868a68-03f3-4968-aba4-8a3591eb97c9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.764597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 318/632] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:35.765991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c64a1-a9f9-441f-9959-ff5473e8f7be None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.766383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c64a1-a9f9-441f-9959-ff5473e8f7be None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.766670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c64a1-a9f9-441f-9959-ff5473e8f7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.786451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-585c64a1-a9f9-441f-9959-ff5473e8f7be None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:35.787741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-585c64a1-a9f9-441f-9959-ff5473e8f7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:35.793445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.794092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.794376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.794652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.805634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.807096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:35.822111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:35.823166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:35.824328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:35.824328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:35.824328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:35.824328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:35.825931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 315/633] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:35.826932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.871559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 319/634] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:35.871559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c64a1-a9f9-441f-9959-ff5473e8f7be None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:35.887457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.947285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc55873-12f2-4a8a-b2ea-2899e5fa4b7d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.964504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:35.977350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c64a1-a9f9-441f-9959-ff5473e8f7be None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:35.984341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:35.985779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:35.986063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:35.986341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:35.987114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:35.987472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:35.988902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.001530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 316/635] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:36.002331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.020303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.020952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.021235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.021513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.032882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.034357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:36.049824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:36.050885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:36.052124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:36.052124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:36.052124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:36.052124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:36.052124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 320/636] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:36.052124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.109897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e463b-9995-4337-ac8b-c65b7cf3c8f7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.112274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.145189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb576c53-663f-4e2a-8209-d0d3e923f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.145811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb576c53-663f-4e2a-8209-d0d3e923f981 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.146095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb576c53-663f-4e2a-8209-d0d3e923f981 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.146412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb576c53-663f-4e2a-8209-d0d3e923f981 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.147108 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb576c53-663f-4e2a-8209-d0d3e923f981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:36.147654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb576c53-663f-4e2a-8209-d0d3e923f981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.149702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb576c53-663f-4e2a-8209-d0d3e923f981 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.152362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68792b-f57e-4b37-8857-2dd174b07c4e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.161040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 317/637] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:36.162225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aa4bd-1496-4fbb-8d2f-43ccb8b5d50c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.162511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aa4bd-1496-4fbb-8d2f-43ccb8b5d50c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.162790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aa4bd-1496-4fbb-8d2f-43ccb8b5d50c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.164020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 318/638] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:36.165223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5aa4bd-1496-4fbb-8d2f-43ccb8b5d50c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.192913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.193821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.194111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.194388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.202110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.205512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:36.210412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:36.213192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 321/639] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:36.214132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.274212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5aa4bd-1496-4fbb-8d2f-43ccb8b5d50c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.309280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504bd50-43d8-4078-babb-0b44c57745c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.309938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504bd50-43d8-4078-babb-0b44c57745c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.310227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504bd50-43d8-4078-babb-0b44c57745c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.310504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504bd50-43d8-4078-babb-0b44c57745c9 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/66f7db7eea94472c82cda44c569d9550/roles/b0563004ff664008b6db05941cf9c9b9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.314025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5c64-5f60-42f4-b4c5-291897462e7c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.325491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504bd50-43d8-4078-babb-0b44c57745c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=66f7db7eea94472c82cda44c569d9550, role_id=b0563004ff664008b6db05941cf9c9b9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.327311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504bd50-43d8-4078-babb-0b44c57745c9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.350737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.351461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.351783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.352066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.352795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:36.353105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.354565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.366747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 322/640] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:36.367943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.388426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 319/641] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:36 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/66f7db7eea94472c82cda44c569d9550/roles/b0563004ff664008b6db05941cf9c9b9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:36.401040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c4db6-00dc-4cf0-b364-a3ad9d2c1d7f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.401040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c4db6-00dc-4cf0-b364-a3ad9d2c1d7f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.401040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c4db6-00dc-4cf0-b364-a3ad9d2c1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.415383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8e1c4db6-00dc-4cf0-b364-a3ad9d2c1d7f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:36.416744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8e1c4db6-00dc-4cf0-b364-a3ad9d2c1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:36.428534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.435810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.480435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53189b4a-3fa2-414e-9169-750d82db39f9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.505407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 320/642] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:36.505407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c4db6-00dc-4cf0-b364-a3ad9d2c1d7f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.550206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.551234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.551366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.551577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.560299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.561773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:36.573882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:36.575092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 323/643] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:36.576246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.628931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c4db6-00dc-4cf0-b364-a3ad9d2c1d7f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.666568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.667257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.667570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.667869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.668619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:36.668948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.670392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.679439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 321/644] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:36.680817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.689285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71799d-0707-457f-a03b-44b93be1cb37 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.690274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.711106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.725887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.726640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.726966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.727236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.734138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.735897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:36.748965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:36.749893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 324/645] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:36.751084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.791102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067926-2493-422f-af37-adf5a9bfe179 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.828102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.828746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.829025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.829297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.840297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:36.846025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:36.847242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:36.848402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:36.855218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 322/646] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:36.855218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.876348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9c618-39a9-4872-97af-1d05b40a6c8d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.911729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:36.912348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:36.912640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:36.912918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:36.913710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:36.914017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:36.916058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:36.928844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:36.935896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 325/647] 10.222.0.49 () {56 vars in 1137 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:36.940544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:36.965958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c4455-78ba-4b1d-b720-4916f3e40805 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:36.984188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.004280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.004960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.005247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.005521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.017281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.018787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:37.035390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:37.036416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:37.037483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:37.037483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:37.037483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:37.037483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:37.037483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 323/648] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:37.037483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.064283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2be4a-1470-4f4f-8ab0-6328799cb592 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.103589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.104517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.105048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.105505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.106554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:37.107085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.108896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.122719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 326/649] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.123569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.155180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6891c-4990-4040-9e14-4655d5d027b7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.175000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.193319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.194053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.194374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.194609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.195466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:37.195813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.197179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.210649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 324/650] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.211557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.227543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.231576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3e4b0-aaa1-4d73-89a5-008eb97c2690 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.270748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.271511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.271793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.272067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.326034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29eae4-470d-465d-8a55-367957625757 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.352330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.353903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.362238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.362944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.363298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.363582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.364355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:37.364717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.366269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.380027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 325/651] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.381048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.429572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.460408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.469536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 327/652] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:37.469536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.498033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48158b-f02c-47a6-9a97-fa5d2f90f1ea None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.543518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.544145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.544487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.544762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.545622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=9072775865604c73ae8361a047945464) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:37.545958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.547573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.554075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 326/653] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/role_assignments?user.id=9072775865604c73ae8361a047945464&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.554694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.604646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5288a-cac4-4bb5-a7a3-f21524068935 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.647847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.648514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.648847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.649125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.661697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.663238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.677735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58533e-7bff-45e7-9a48-ce3d48643cb4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.681447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:37.683566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:37.684456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:37.685419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:37.685419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:37.685419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:37.685419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:37.691494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 328/654] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:37.692716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.717443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.718107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.718393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.718672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.728467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.729950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:37.743584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:37.744571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:01:37.745567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:37.750439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 327/655] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:37.751379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.755835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:37.824313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f68aa4-bb00-4795-a10b-8b67c02173a9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.869026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.870612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.870894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.871168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.879672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadabfdd-d6d7-4682-8038-599fc3a04849 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:37.886763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.889034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:37.909395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:37.910341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:37.911267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:37.911267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:37.911267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:37.911267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:37.912641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 329/656] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:37.917585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.924796 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:37.925741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:37.926228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:37.926720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:37.927545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:37.927946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:37.929372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:37.942158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 328/657] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:37.947271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:37.969749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.049777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aba3ff-55e8-4a27-a335-c3b7db2ffd0c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.054543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5db3-538b-42a9-899d-1bce7888b0f0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.057303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.096756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.097376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.097660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.097935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.098715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.098931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.100349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.104768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.105644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.106122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.106591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.108204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.108764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.111253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.117088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 329/658] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.118789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.135662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 330/659] 10.222.0.49 () {56 vars in 1137 bytes} [Tue Aug 2 10:01:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.137490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.232452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.250643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c0d8b-31ea-48be-9955-031328bdc26b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.250962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30712d30-948a-40ec-be96-88191a2adc4e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.292747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.293852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.294360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.294909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.296798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.297549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.297930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.298301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.305504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.306525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.307465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.308237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:38.320622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:38.322277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:01:38.323302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:38.323860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:38.324636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:38.325510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:38.327797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 330/660] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:38.328619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 331/661] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:38.329160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.329692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.352961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.436647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7058658-a0a8-4d3f-a9d8-45c7263cf4b8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.469382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c00a4d-00fc-48e9-80b9-bd84eb815e17 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.483595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.484313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.484725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.485012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.485763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.486076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.487534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.500000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 332/662] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.500784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.515099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.515846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.516215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.516497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.535483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.587742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.591325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.592913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.607167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee425-cab6-45a6-a595-725883e8c2b9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.650793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b7aec-84cc-49b7-a6ec-7e0f271ddf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.651884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b7aec-84cc-49b7-a6ec-7e0f271ddf78 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.652175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b7aec-84cc-49b7-a6ec-7e0f271ddf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.652741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b7aec-84cc-49b7-a6ec-7e0f271ddf78 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.653810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b7aec-84cc-49b7-a6ec-7e0f271ddf78 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.654145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b7aec-84cc-49b7-a6ec-7e0f271ddf78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.656248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b7aec-84cc-49b7-a6ec-7e0f271ddf78 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.683709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 333/663] 10.222.0.49 () {56 vars in 1139 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.683709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5143536-0c5c-45a5-b0f8-a933bc0d1a48 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.683709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5143536-0c5c-45a5-b0f8-a933bc0d1a48 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.683709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5143536-0c5c-45a5-b0f8-a933bc0d1a48 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.684509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 334/664] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:38.685319 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5143536-0c5c-45a5-b0f8-a933bc0d1a48 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.701814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 331/665] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:38.702590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.751448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.797972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5143536-0c5c-45a5-b0f8-a933bc0d1a48 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.810146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66f0ce-00fd-499f-9618-73f999822efd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:38.835995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb758ccb-b831-40ae-9305-87ed1b67451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.837152 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb758ccb-b831-40ae-9305-87ed1b67451a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.837433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb758ccb-b831-40ae-9305-87ed1b67451a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.837877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb758ccb-b831-40ae-9305-87ed1b67451a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.838992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb758ccb-b831-40ae-9305-87ed1b67451a None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=2dbb92bcc5054287a96fbfac03d2975a) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:38.839340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb758ccb-b831-40ae-9305-87ed1b67451a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.841213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb758ccb-b831-40ae-9305-87ed1b67451a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.859220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 335/666] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/role_assignments?user.id=2dbb92bcc5054287a96fbfac03d2975a&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:38.859220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6099-2bc3-4ee8-bac9-8e18fb8124b0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.859220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6099-2bc3-4ee8-bac9-8e18fb8124b0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.859220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6099-2bc3-4ee8-bac9-8e18fb8124b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.860997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:38.861729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:38.862089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:38.862439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:38.864659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:38.878163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3bba6099-2bc3-4ee8-bac9-8e18fb8124b0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:38.879644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3bba6099-2bc3-4ee8-bac9-8e18fb8124b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:38.937596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:38.939238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:38.983374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 336/667] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:38.984833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6099-2bc3-4ee8-bac9-8e18fb8124b0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:38.996559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.080390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 332/668] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:39.081642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.129116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6099-2bc3-4ee8-bac9-8e18fb8124b0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.161220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.168082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.168991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.169464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.169949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.180897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.182649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:39.200901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:39.201846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:39.202770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:39.202770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:39.202770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:39.202770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:39.204494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 337/669] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:39.206755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9af39-37b3-4600-803e-d3d51e7fe4be None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.207730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.251174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.252483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.252918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.253466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.265784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:39.271324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:39.272388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:39.273255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:39.277005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 333/670] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:39.278091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.284333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.349547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf94434-23b6-4733-98cf-953db1cd4a16 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.400052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.401336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.402054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.402525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.415409 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6495f-ca42-459b-b740-30a3ac7dcc17 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.417630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.419223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:39.434668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:39.435686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:39.436957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:39.436957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:39.436957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:39.436957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:39.436957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 338/671] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:39.437603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.458933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.459915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.460397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.460920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.462477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:39.463017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.464734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.475606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.480376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 334/672] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:39.481620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.548823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c0a2d-9ef4-4933-a809-733855ab6adc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.559299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.587749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.588644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.589076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.589513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.590489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:39.590942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.592662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.607096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eaf950-a99b-4b2a-94ab-b05810a4bc36 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.607476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 339/673] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:39.608589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.641817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.642490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.642877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.643115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.643952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:39.644399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.645681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.660638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 335/674] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:39.661670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.721349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c868c-f436-4f9b-b7a6-badfacc87290 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.723513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.760081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.760882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.761279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.761571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.770237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b525ee-f051-4ff2-8f37-f3c422a370c1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.770446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.771952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:39.789210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:39.790959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 340/675] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:39.792786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.800118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:39.811042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b0f3f-64ee-4e13-af29-471b4edee1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.811737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b0f3f-64ee-4e13-af29-471b4edee1ea None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.812049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b0f3f-64ee-4e13-af29-471b4edee1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.812339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b0f3f-64ee-4e13-af29-471b4edee1ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.823434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b0f3f-64ee-4e13-af29-471b4edee1ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.824896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b0f3f-64ee-4e13-af29-471b4edee1ea None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2a0b0f3f-64ee-4e13-af29-471b4edee1ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:39.840291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:39.841431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:39.842731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:39.842731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:39.842731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:39.842731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:39.842731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 336/676] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:39.842731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf2b53-a7bf-45aa-88fa-8e6aca014e24 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.843146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf2b53-a7bf-45aa-88fa-8e6aca014e24 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.843146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf2b53-a7bf-45aa-88fa-8e6aca014e24 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.844677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 337/677] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:39.845771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf2b53-a7bf-45aa-88fa-8e6aca014e24 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.912986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87209b6-4eef-4a7a-9ce1-d5dc917bfc21 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.953519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.954251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.954540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.954816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.956661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf2b53-a7bf-45aa-88fa-8e6aca014e24 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:39.962507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.963981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:39.975387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:39.976369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:39.977356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:39.981294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 341/678] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:39.982138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:39.993487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a584-72f4-4f17-99f2-6c3c0ffcee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:39.994106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a584-72f4-4f17-99f2-6c3c0ffcee20 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:39.994590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a584-72f4-4f17-99f2-6c3c0ffcee20 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:39.994712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a584-72f4-4f17-99f2-6c3c0ffcee20 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:39.995445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a584-72f4-4f17-99f2-6c3c0ffcee20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:39.995758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a584-72f4-4f17-99f2-6c3c0ffcee20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:39.997199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a584-72f4-4f17-99f2-6c3c0ffcee20 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.003756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.010975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 338/679] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:40.011994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26687273-40f0-49c3-bc2d-6c11a3429f41 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.012283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26687273-40f0-49c3-bc2d-6c11a3429f41 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.012587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26687273-40f0-49c3-bc2d-6c11a3429f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.032771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-26687273-40f0-49c3-bc2d-6c11a3429f41 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:40.034029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-26687273-40f0-49c3-bc2d-6c11a3429f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:40.064608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.104771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210999d-1d27-4e75-9d77-60368dbf98c6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.115917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 339/680] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:40.117166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26687273-40f0-49c3-bc2d-6c11a3429f41 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.144460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.145548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.145548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.145741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.146395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:40.146698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.148245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.166005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 342/681] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:40.166875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.231204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26687273-40f0-49c3-bc2d-6c11a3429f41 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.250825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.272950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4188ba-49c7-4fb9-8453-7c4cefe5e426 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.276677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.277322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.277608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.277885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.278643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:40.278952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.280443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.299488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 340/682] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:40.300800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.318486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.319569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.320336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.321427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.322452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=700720d6f66e4de2a65b808afce70e39) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:40.323053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.325420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.329012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.334205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 343/683] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/role_assignments?user.id=700720d6f66e4de2a65b808afce70e39&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:40.339095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.410711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287610a8-31f0-4369-a654-96aab69239c7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.454430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.455231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.455521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.455801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.468039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:40.473687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:40.474686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:40.475594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:40.479619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 341/684] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:40.480592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.497436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72945d-0d04-4c5a-939e-83fc15d6d3c1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.546949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.547638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.547923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.548203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.557488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.559041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:40.572543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:40.573509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:40.574445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 344/685] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:40.575063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.582495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.599777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.619270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971183-7037-49c7-9ba5-0973c2bd3949 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.666859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.667525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.667815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.668096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.668906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=09cabb184e704aecbb0a3bfb1cd0a584) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:40.669220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.670667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.683644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 342/686] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/role_assignments?user.id=09cabb184e704aecbb0a3bfb1cd0a584&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:40.683644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.734606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9634-4886-45f4-b616-d659d4e9a5d2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.789685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.790831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.791497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.791991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.793216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:40.793740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.796015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.809301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 345/687] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:40.818301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.869685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7f20e-230b-4b08-9464-891300887f0b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.908915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.909762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.910060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.910429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.911330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:40.911713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.913321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.932021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.939348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 343/688] 10.222.0.49 () {56 vars in 1135 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:40.941105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.952912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.976290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2994e-128e-442a-b4ca-49a24eb172ef None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.015023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.016075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.016663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.017123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.030698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.032519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:41.049761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:41.050693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:41.051659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:41.051659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:41.051659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:41.051659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:41.053784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 346/689] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:41.054337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8ba40-a3ad-4e54-b56e-df3b863795ef None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.062963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.095533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfce64c-1763-4b94-96bf-9f5d2a177b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.096263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce64c-1763-4b94-96bf-9f5d2a177b4c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.096551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce64c-1763-4b94-96bf-9f5d2a177b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.096830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce64c-1763-4b94-96bf-9f5d2a177b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.184623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfce64c-1763-4b94-96bf-9f5d2a177b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.185993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2195dd-32ab-4914-899e-58e4e72332a1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.187030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfce64c-1763-4b94-96bf-9f5d2a177b4c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.202512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.203870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.223295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37914d36-7027-4291-b83a-8474e591dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.224053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914d36-7027-4291-b83a-8474e591dfbd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.224281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914d36-7027-4291-b83a-8474e591dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.224587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914d36-7027-4291-b83a-8474e591dfbd None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.225289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37914d36-7027-4291-b83a-8474e591dfbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:41.225610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37914d36-7027-4291-b83a-8474e591dfbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.227699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37914d36-7027-4291-b83a-8474e591dfbd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.246525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 347/690] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:41.248197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eecf5-73c9-4470-ad12-49e7220cfff2 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.248503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eecf5-73c9-4470-ad12-49e7220cfff2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.248784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eecf5-73c9-4470-ad12-49e7220cfff2 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.250140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 348/691] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:41.256146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eecf5-73c9-4470-ad12-49e7220cfff2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.324301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 344/692] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:01:41.325686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429135fa-dd7e-418d-84d5-82363a3bc504 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.326046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429135fa-dd7e-418d-84d5-82363a3bc504 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.326354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429135fa-dd7e-418d-84d5-82363a3bc504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.344890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-429135fa-dd7e-418d-84d5-82363a3bc504 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:41.346187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-429135fa-dd7e-418d-84d5-82363a3bc504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:41.387933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eecf5-73c9-4470-ad12-49e7220cfff2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.427210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 345/693] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:41.427210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429135fa-dd7e-418d-84d5-82363a3bc504 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.449866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.450447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.450730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.450997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.458874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.460426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:41.473183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:41.474147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:41.475069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:41.479789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 349/694] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:41.480646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.480902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.524282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.560879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429135fa-dd7e-418d-84d5-82363a3bc504 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.603382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.605053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.605689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.606266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.607887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e47305-d945-4895-ad7c-e5750af13224 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.619280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.621028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:41.637453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:41.638421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:41.639343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:41.639343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:41.639343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:41.639343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:41.641815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 346/695] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:41.643319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.646395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.647198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.647584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.647931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.650043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.650704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:41.663106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:41.664177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:41.665082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:41.669082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 350/696] 10.222.0.49 () {56 vars in 1129 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:41.669941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.757136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.768184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacf2a3-e8a5-47d1-b950-d8935a4d74c3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.776242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:41.779827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e840-54dc-4330-8776-eaa71ab17da4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.803736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.805897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.806160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.806438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.807283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:41.809568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.809568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.816954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.817869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.818332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.818795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.819740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:41.820269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.822446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.829041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 347/697] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:41.832619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.835735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 351/698] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:41.837018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.935610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4d99e-0dac-4f56-bc3d-4b8d1ad46ede None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.944422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9d308-e157-49d4-a9bc-5835073bb466 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:41.972132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.972771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.973060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.973343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.974033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:41.974569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:41.975779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:41.985772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 348/699] 10.222.0.49 () {56 vars in 1124 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:41.985772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:41.988378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:41.989268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:41.989765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:41.990250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:41.998214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.021094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.064961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.066784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.091467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bf779-e9c0-4e9d-a764-2bc9c3dafbc9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.142624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.143304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.143589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.143863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.151459 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.152924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:42.167740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:42.168708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:42.169631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 349/700] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:42.170113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.186746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 352/701] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:42.188399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.250662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.253213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.282533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd220-0aaf-48f0-a276-9653f782d1ef None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.295927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44007bb5-4e61-4d4c-b5f7-9612b564e179 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.321204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.321646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.321933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.322206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.322933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:42.323179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.324690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.339243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 350/702] 10.222.0.49 () {56 vars in 1124 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:42.339243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.343077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.344146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.344678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.345163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.346148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:42.346675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.348419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.371362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 353/703] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:42.377565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.452786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82220c-698e-40f4-9156-be73748021a8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.493641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.494351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.494630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.494911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.498423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.509941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.513287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.539292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e685b0-a757-4bb2-8b6b-bdb0c7ae7356 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:42.552340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:42.553322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:42.554196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:42.554196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:42.554196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:42.554196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:42.554196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 351/704] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:42.560430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.561164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.582884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb2f2e-60ec-46a4-9a2d-7bf662fd8191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.583618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2f2e-60ec-46a4-9a2d-7bf662fd8191 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.583906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2f2e-60ec-46a4-9a2d-7bf662fd8191 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.584185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2f2e-60ec-46a4-9a2d-7bf662fd8191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.584988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb2f2e-60ec-46a4-9a2d-7bf662fd8191 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=66f7db7eea94472c82cda44c569d9550) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:42.585303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb2f2e-60ec-46a4-9a2d-7bf662fd8191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.586995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb2f2e-60ec-46a4-9a2d-7bf662fd8191 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.594338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 354/705] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/role_assignments?user.id=66f7db7eea94472c82cda44c569d9550&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:42.646219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b307b-8375-4393-b341-9a866db23773 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.646514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b307b-8375-4393-b341-9a866db23773 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.646802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b307b-8375-4393-b341-9a866db23773 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.648478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 355/706] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:42.656251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad172-6bf9-42e2-8ef6-8be13ae2aaa8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.656544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad172-6bf9-42e2-8ef6-8be13ae2aaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.656828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad172-6bf9-42e2-8ef6-8be13ae2aaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.680873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-772ad172-6bf9-42e2-8ef6-8be13ae2aaa8 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:42.682763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-772ad172-6bf9-42e2-8ef6-8be13ae2aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:42.731680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d151507-d174-4a4a-95d8-7dcca5a4fce5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.793850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:42.794792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:42.795307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:42.795787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:42.796758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:42.797250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:42.798880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:42.802540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 356/707] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:42.817905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 352/708] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:42.821596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad172-6bf9-42e2-8ef6-8be13ae2aaa8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.827645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:42.931617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:42.965823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad172-6bf9-42e2-8ef6-8be13ae2aaa8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.973332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe7f92-d92e-473d-b95c-a51729e4ebb5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:42.987453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:43.021729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.022649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.022965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.023340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.026097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.027355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.027669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.027955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.033113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.035054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.042567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.044313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:43.054873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:43.056678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:43.057574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:43.063519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 357/709] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:43.065245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:43.066152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:43.067039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:43.067039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:43.067039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:43.067039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:43.067321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:43.067398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 353/710] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:43.075540 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:43.199886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:43.203865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341b053-4ac2-47a5-babe-247b463ffc65 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:43.213981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf2a57-5c9a-4434-83a4-ea3770e59aed None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:43.253018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2942b2-83a6-4c3a-bdc8-01e5ae415708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.254089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2942b2-83a6-4c3a-bdc8-01e5ae415708 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.254390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2942b2-83a6-4c3a-bdc8-01e5ae415708 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.254672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2942b2-83a6-4c3a-bdc8-01e5ae415708 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.255844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2942b2-83a6-4c3a-bdc8-01e5ae415708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:43.256164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2942b2-83a6-4c3a-bdc8-01e5ae415708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.257851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2942b2-83a6-4c3a-bdc8-01e5ae415708 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.263471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500233cf-9f1c-43d8-a7e6-03587248d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.264416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500233cf-9f1c-43d8-a7e6-03587248d964 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.264900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500233cf-9f1c-43d8-a7e6-03587248d964 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.265384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500233cf-9f1c-43d8-a7e6-03587248d964 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.266458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500233cf-9f1c-43d8-a7e6-03587248d964 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:43.266971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500233cf-9f1c-43d8-a7e6-03587248d964 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.268820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500233cf-9f1c-43d8-a7e6-03587248d964 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.277391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 354/711] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:43.277391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0d881-d9e8-48b0-bd7c-f88f944717ec None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.277391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0d881-d9e8-48b0-bd7c-f88f944717ec None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.277391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0d881-d9e8-48b0-bd7c-f88f944717ec None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.277391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 355/712] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:43.278613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0d881-d9e8-48b0-bd7c-f88f944717ec None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:43.293134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 358/713] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:43.294891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26169ec6-2a65-49de-bc6a-644adebae6a6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.295366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26169ec6-2a65-49de-bc6a-644adebae6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.295849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26169ec6-2a65-49de-bc6a-644adebae6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.320647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-26169ec6-2a65-49de-bc6a-644adebae6a6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:43.322375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-26169ec6-2a65-49de-bc6a-644adebae6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:43.362402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:43.401299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0d881-d9e8-48b0-bd7c-f88f944717ec None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:43.419406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 359/714] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:43.420451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26169ec6-2a65-49de-bc6a-644adebae6a6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:43.441029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.441656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.441942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.442221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.442900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.444372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.531151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26169ec6-2a65-49de-bc6a-644adebae6a6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:43.548042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 356/715] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:01:43 2022] POST /identity/v3/users => generated 254 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:43.549712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:43.571330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.572152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:43.572398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.573480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.573783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.574707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=9072775865604c73ae8361a047945464) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:43.575034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.576604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.583487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 360/716] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:43 2022] GET /identity/v3/role_assignments?user.id=9072775865604c73ae8361a047945464&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:43.725516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deeed1-606e-424f-a91b-2e13ef0be38b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:43.777526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.778203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.778487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.778761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.779523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.781093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:43.793750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:43.795025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:43.796076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 357/717] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:43 2022] GET /identity/v3/services/network => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:43.802642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:43.835501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:43.917045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba677f-b399-46e4-8f78-8a35e2c0347c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:43.952146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333cfcd-60f1-4535-bb02-f2a1ec233799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:43.952775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333cfcd-60f1-4535-bb02-f2a1ec233799 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:43.953069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333cfcd-60f1-4535-bb02-f2a1ec233799 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:43.953341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333cfcd-60f1-4535-bb02-f2a1ec233799 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:43.954014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333cfcd-60f1-4535-bb02-f2a1ec233799 None admin] RBAC: Adding query filter params (name=network) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:43.954370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333cfcd-60f1-4535-bb02-f2a1ec233799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:43.955805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333cfcd-60f1-4535-bb02-f2a1ec233799 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:43.960768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 361/718] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:01:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:43.966512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:44.077347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:44.097430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95dcf1-cd97-4d25-af60-71c7eb0654d5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:44.140883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a21bb0-b033-48dd-b331-89e8819b92e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:44.141889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a21bb0-b033-48dd-b331-89e8819b92e8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:44.142380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a21bb0-b033-48dd-b331-89e8819b92e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:44.142870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a21bb0-b033-48dd-b331-89e8819b92e8 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:44.143800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a21bb0-b033-48dd-b331-89e8819b92e8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:44.144311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a21bb0-b033-48dd-b331-89e8819b92e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:44.145922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a21bb0-b033-48dd-b331-89e8819b92e8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:44.152112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 358/719] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:01:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:44.835557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f92ef7-e3d5-4ba4-a326-fe1bc279be97 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:44.836068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f92ef7-e3d5-4ba4-a326-fe1bc279be97 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:44.836068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f92ef7-e3d5-4ba4-a326-fe1bc279be97 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:44.837707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 362/720] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:44.842711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79fa91-46e7-40f1-bb10-713f48e552c6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:44.843155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79fa91-46e7-40f1-bb10-713f48e552c6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:44.843382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79fa91-46e7-40f1-bb10-713f48e552c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:44.868860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ed79fa91-46e7-40f1-bb10-713f48e552c6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:44.870093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ed79fa91-46e7-40f1-bb10-713f48e552c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:45.068659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 359/721] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:45.104022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f92ef7-e3d5-4ba4-a326-fe1bc279be97 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:45.237588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:45.252876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f92ef7-e3d5-4ba4-a326-fe1bc279be97 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:45.309891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615171a-8017-4871-86ae-f5c00398a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:45.310527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615171a-8017-4871-86ae-f5c00398a2e2 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.310817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615171a-8017-4871-86ae-f5c00398a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.311098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615171a-8017-4871-86ae-f5c00398a2e2 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.311841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615171a-8017-4871-86ae-f5c00398a2e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:45.313207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615171a-8017-4871-86ae-f5c00398a2e2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:45.422746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 363/722] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:01:45 2022] POST /identity/v3/services => generated 251 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:45.518427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2260ec8-54cf-4e61-979b-7314819cc183 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.518764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2260ec8-54cf-4e61-979b-7314819cc183 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.519073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2260ec8-54cf-4e61-979b-7314819cc183 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.520919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 360/723] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:45.527590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224db6-29f6-48b4-b19d-fb34a8ef3666 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.529407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224db6-29f6-48b4-b19d-fb34a8ef3666 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.529888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224db6-29f6-48b4-b19d-fb34a8ef3666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.560447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c5224db6-29f6-48b4-b19d-fb34a8ef3666 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:45.561694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c5224db6-29f6-48b4-b19d-fb34a8ef3666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:45.667703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 364/724] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:45.691049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2260ec8-54cf-4e61-979b-7314819cc183 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:45.791554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:45.824404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2260ec8-54cf-4e61-979b-7314819cc183 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:45.869684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3fe47-7570-4cc0-9ffc-b1cdb09c80db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:45.870495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3fe47-7570-4cc0-9ffc-b1cdb09c80db None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:45.870844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3fe47-7570-4cc0-9ffc-b1cdb09c80db None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:45.871132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3fe47-7570-4cc0-9ffc-b1cdb09c80db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:45.886088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3fe47-7570-4cc0-9ffc-b1cdb09c80db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8bc3fe47-7570-4cc0-9ffc-b1cdb09c80db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:45.891064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:45.892065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:45.892909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 361/725] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:45.900771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224db6-29f6-48b4-b19d-fb34a8ef3666 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.022809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.051115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224db6-29f6-48b4-b19d-fb34a8ef3666 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.098578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.099237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.099531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.099812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.100572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:46.100959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.102984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.107771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427991-0f5f-462a-bf8d-0c528935e193 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.108106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427991-0f5f-462a-bf8d-0c528935e193 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.108339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427991-0f5f-462a-bf8d-0c528935e193 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.110490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 362/726] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:46.115463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa195c-ca47-47a4-82a8-d604142ce6fb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.115755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa195c-ca47-47a4-82a8-d604142ce6fb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.116038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa195c-ca47-47a4-82a8-d604142ce6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.117521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 365/727] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:46.123696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.150909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-67aa195c-ca47-47a4-82a8-d604142ce6fb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:46.152153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-67aa195c-ca47-47a4-82a8-d604142ce6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:46.262219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 363/728] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:01:46.293946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa195c-ca47-47a4-82a8-d604142ce6fb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.317817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de59040-e254-47bc-be26-90a2e7520219 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.385386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c9e94-795b-40d6-af69-9f54427c42b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.386076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c9e94-795b-40d6-af69-9f54427c42b6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.386365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c9e94-795b-40d6-af69-9f54427c42b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.386641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c9e94-795b-40d6-af69-9f54427c42b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.388899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.409442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.411789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c9e94-795b-40d6-af69-9f54427c42b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.413263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c9e94-795b-40d6-af69-9f54427c42b6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7d0c9e94-795b-40d6-af69-9f54427c42b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:46.433934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:46.435304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:46.436374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:46.436374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:46.436374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:46.436374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:46.438493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa195c-ca47-47a4-82a8-d604142ce6fb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.439590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 366/729] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:46.440873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77634dcf-ca79-4bc2-8b5a-b7f61d537f8d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.441164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77634dcf-ca79-4bc2-8b5a-b7f61d537f8d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.441444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77634dcf-ca79-4bc2-8b5a-b7f61d537f8d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.442899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 367/730] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:46.443452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77634dcf-ca79-4bc2-8b5a-b7f61d537f8d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.487541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ba4d2-7b89-4a81-a5b0-b1ba3af7a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.488221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ba4d2-7b89-4a81-a5b0-b1ba3af7a155 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.488506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ba4d2-7b89-4a81-a5b0-b1ba3af7a155 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.488781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ba4d2-7b89-4a81-a5b0-b1ba3af7a155 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.506337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ba4d2-7b89-4a81-a5b0-b1ba3af7a155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3f9ba4d2-7b89-4a81-a5b0-b1ba3af7a155 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:46.512910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:46.513880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:46.514739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:46.516505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 364/731] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:46.517687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad8710-7e89-4d65-89b9-807f731a4447 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.517968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad8710-7e89-4d65-89b9-807f731a4447 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.518256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad8710-7e89-4d65-89b9-807f731a4447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.542574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e2ad8710-7e89-4d65-89b9-807f731a4447 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:46.544093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e2ad8710-7e89-4d65-89b9-807f731a4447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:46.579992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77634dcf-ca79-4bc2-8b5a-b7f61d537f8d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.616754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.617746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.618778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.619162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.619987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:46.620313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 365/732] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:46.620578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.621181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad8710-7e89-4d65-89b9-807f731a4447 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.622292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.636607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 368/733] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:46.641718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.686910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.703674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.740083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad8710-7e89-4d65-89b9-807f731a4447 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.755003 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdace5-6554-43f2-9e92-b3530ff596df None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.776468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.777094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.777389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.777669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.778414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:46.778727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.780161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.789286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 366/734] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:46.790648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.800759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccd6ea-1a44-4ad0-9958-b5d1effa5350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.800759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccd6ea-1a44-4ad0-9958-b5d1effa5350 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.800759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccd6ea-1a44-4ad0-9958-b5d1effa5350 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.800759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccd6ea-1a44-4ad0-9958-b5d1effa5350 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.807677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccd6ea-1a44-4ad0-9958-b5d1effa5350 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:46.809136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccd6ea-1a44-4ad0-9958-b5d1effa5350 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:46.903680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99011e74-4a89-4699-ad58-e225163f1c4f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:46.923839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4cccd6ea-1a44-4ad0-9958-b5d1effa5350 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:46.936428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:46.937778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:46.938701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 369/735] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:46.939965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000105-d99a-4a90-875f-eeeb411649ac None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.939965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000105-d99a-4a90-875f-eeeb411649ac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.939965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000105-d99a-4a90-875f-eeeb411649ac None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.941214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 370/736] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:46.941893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77000105-d99a-4a90-875f-eeeb411649ac None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:46.952933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:46.953690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:46.953983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:46.954263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:46.966593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:46.971873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:46.972833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:46.973650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:46.976040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 367/737] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:46.977060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.061992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77000105-d99a-4a90-875f-eeeb411649ac None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.075970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.082761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a845e7-7cc2-4aaf-91ab-3cd260ee33a2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.097239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d0dfa-1f52-415b-87f0-7f3df2dff5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.097879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0dfa-1f52-415b-87f0-7f3df2dff5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.098167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0dfa-1f52-415b-87f0-7f3df2dff5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.098445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0dfa-1f52-415b-87f0-7f3df2dff5f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.108478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d0dfa-1f52-415b-87f0-7f3df2dff5f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.109909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d0dfa-1f52-415b-87f0-7f3df2dff5f0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.119386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.120623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.121085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.121561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.122537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:47.123023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.124762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-276d0dfa-1f52-415b-87f0-7f3df2dff5f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:47.126241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:47.127081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 371/738] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43189d97-620b-4f38-bcbf-374621245f7a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43189d97-620b-4f38-bcbf-374621245f7a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.132804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43189d97-620b-4f38-bcbf-374621245f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.151394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 368/739] 10.222.0.49 () {56 vars in 1135 bytes} [Tue Aug 2 10:01:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.151394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.151676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-43189d97-620b-4f38-bcbf-374621245f7a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:47.151676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-43189d97-620b-4f38-bcbf-374621245f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:47.186120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.235212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 372/740] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:01:47.235212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43189d97-620b-4f38-bcbf-374621245f7a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.255137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d988d-6d7f-437c-a40f-2296881d6c6b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.294043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.294802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.295295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.295426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.296227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:47.296486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.297936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.306458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 369/741] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.307264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.338403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.345195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43189d97-620b-4f38-bcbf-374621245f7a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.389666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.390676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.390953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.391248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.391941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:47.392242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.393867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.405559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 373/742] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:47.410699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.417888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.424683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4fbcf-3719-4336-b22c-8c5792573b36 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.465547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.466280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.466851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.466851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.534352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c038a50-9b9c-4cca-96cb-78038ca9f75e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.556549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.558022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.575438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.576242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.576515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.576790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.577633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.579072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.653088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.659265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 370/743] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:01:47.660068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.670037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 374/744] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:01:47 2022] POST /identity/v3/services => generated 240 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:47.671024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.779628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.802778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422dacf-3a17-40f4-9781-f272b9622270 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.832324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636df0f-d543-4728-8b90-2f0ffa9cfd91 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:47.861346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.862008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.862295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.862571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.875307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.876781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.877125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe5cfd-5f25-4a5d-99f8-01f2cda673ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:47.877831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe5cfd-5f25-4a5d-99f8-01f2cda673ed None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.878118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe5cfd-5f25-4a5d-99f8-01f2cda673ed None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.878395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe5cfd-5f25-4a5d-99f8-01f2cda673ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.887083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe5cfd-5f25-4a5d-99f8-01f2cda673ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:47.888589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe5cfd-5f25-4a5d-99f8-01f2cda673ed None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:47.896231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:47.897189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:47.898189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:47.898189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:47.898189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:47.898189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:47.899037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 371/745] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:47.899808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-afbe5cfd-5f25-4a5d-99f8-01f2cda673ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:47.900963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:47.901874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:47.903299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 375/746] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:47.909540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0578d-f356-48c6-9ebd-c06f0794efb1 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:47.909540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0578d-f356-48c6-9ebd-c06f0794efb1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:47.909540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0578d-f356-48c6-9ebd-c06f0794efb1 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:47.909540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 376/747] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:47.909540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0578d-f356-48c6-9ebd-c06f0794efb1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:47.964746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:47.987660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.020851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594de9b-3575-47d3-81a9-718b3a377b0c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.021977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0578d-f356-48c6-9ebd-c06f0794efb1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.057527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.058150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.058439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.058718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.059444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.059758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.060833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7100b13-d095-491f-916d-fedc611e0b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.061121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.061853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7100b13-d095-491f-916d-fedc611e0b33 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.062325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7100b13-d095-491f-916d-fedc611e0b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.062752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7100b13-d095-491f-916d-fedc611e0b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.075241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 377/748] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.075414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.080265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7100b13-d095-491f-916d-fedc611e0b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.081740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7100b13-d095-491f-916d-fedc611e0b33 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d7100b13-d095-491f-916d-fedc611e0b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:48.096629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:48.098513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:48.100639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:48.100639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:48.100639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:48.100639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:48.102581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 372/749] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:48.104036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38fd82-13e6-49a1-9342-738f81062cbe None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.104308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38fd82-13e6-49a1-9342-738f81062cbe None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.104589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38fd82-13e6-49a1-9342-738f81062cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.123553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7c38fd82-13e6-49a1-9342-738f81062cbe None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:48.125035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c38fd82-13e6-49a1-9342-738f81062cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:48.185638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27489b90-e439-402d-9133-85f0ab829cec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.203357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 373/750] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:48.203357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38fd82-13e6-49a1-9342-738f81062cbe None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.221918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.228141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.236899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.236899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.236899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.236899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.236899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.236899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.237312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.249630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 378/751] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.251169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.316846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38fd82-13e6-49a1-9342-738f81062cbe None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.359336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.360050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.360344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.360628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.361274 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f962e8-08f1-4205-9b22-2ff8f276267c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.369411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.371015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:48.384231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:48.385206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:48.386142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 374/752] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:48.386753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.402865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.403746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.404194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.404676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.405644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.406095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.407857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.421112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 379/753] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.422106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.474270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.487071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.497427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b0fa8-13a8-4326-92b2-fc823bffbdd1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.531724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.532373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.532657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.532939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.533567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.534956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.540819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aaf17-2c92-4e47-8b0c-2c3a07565756 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:48.545127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:48.546278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:48.547353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:48.551360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 375/754] 10.222.0.49 () {56 vars in 1129 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:48.552286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.581524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.582171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.582456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.582772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.595096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.596566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:48.611729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:48.612632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:48.613537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:48.613537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:48.613537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:48.613537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:48.613537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 380/755] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:48.614373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.668347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a8863-7bb8-488f-820e-10d40794e438 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.704212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.704841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.705127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.705404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.706196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.706504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.707966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.721096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.724196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0676-fd9e-4499-a399-188f9e71264b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.735007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 376/756] 10.222.0.49 () {56 vars in 1149 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.735007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.745606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.766897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.767637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.767927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.768209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.776239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.777745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:48.791203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:48.792257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:48.793440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:48.798973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 381/757] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:48.798973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.866177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99afde5-8340-47cf-95e7-e67d5d901f04 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.903843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.904469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.904752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.905033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.905837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.906145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.907589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.913330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 377/758] 10.222.0.49 () {56 vars in 1124 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.914195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.940442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7fdcf-09dd-483c-b585-ae72454f0089 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.975498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.976113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.976461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.976664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.977375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:48.977685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.979318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.997024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 382/759] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:48.997024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.998346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.023594 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.026495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2289f-98e3-47f3-87cd-af6cf06a8f0b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.065473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.066195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.066491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.066781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.116856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1cffc-b8da-4e61-bc5b-1737c5ea74ec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.142509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.143991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.153933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.154564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.154855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.155138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.155915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:49.156229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.157691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.170202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 383/760] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.171246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.246015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.266491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.276267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 378/761] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:01:49.281616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.287808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e7fea-092e-49a0-b8e5-86939d1136aa None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.324868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.325529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.325839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.326120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.326858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:49.327176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.328774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.340439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 384/762] 10.222.0.49 () {56 vars in 1124 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.340439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.401449 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf993e-70f4-4dd1-aff3-24ca0ef12463 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.441018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.441819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.442214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.442520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.445345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e450b-32c9-41fe-8dfb-5dae22dcfdb1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.450091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.452418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:49.477339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:49.478978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 379/763] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:49.485047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.489184 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.489828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.490107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.490385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.491180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=66f7db7eea94472c82cda44c569d9550) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:49.491528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.492963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.511229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 385/764] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/role_assignments?user.id=66f7db7eea94472c82cda44c569d9550&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.511229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.523900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.548887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.632131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1013-d71a-489d-8ca8-c981b505fa32 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.652043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4be34-098e-4e84-bf1a-aaeb4a72aef3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.680100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.681160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.681654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.682158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.696834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.698839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.709907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.711981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.713366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.714078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.715507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] RBAC: Adding query filter params (interface=public, service_id=ebaef66f920c4f8587b71a153579be2a, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:49.715930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.717421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:49.718238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:49.719139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:49.720055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:49.720055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:49.720055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:49.720055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:49.726418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 380/765] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:49.726418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.734251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 386/766] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/endpoints?service_id=ebaef66f920c4f8587b71a153579be2a&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.734251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.813361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.829859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:49.846068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52ba3f-1f1d-4471-996e-6412549f2e49 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.847441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7eedc-7fcd-4a79-88e9-4bc0e3f562aa None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.882427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07995a56-5af6-4117-9e10-c980d72a67d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.883336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07995a56-5af6-4117-9e10-c980d72a67d5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.883624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07995a56-5af6-4117-9e10-c980d72a67d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.884417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07995a56-5af6-4117-9e10-c980d72a67d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.885223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07995a56-5af6-4117-9e10-c980d72a67d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:49.885638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07995a56-5af6-4117-9e10-c980d72a67d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.886624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.887017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07995a56-5af6-4117-9e10-c980d72a67d5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.887283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.887573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.887855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.888611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:49.888926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.890372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.899111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 387/767] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.901005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831676-8ab3-4434-8ff2-a4a657770227 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.901296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831676-8ab3-4434-8ff2-a4a657770227 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.901573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831676-8ab3-4434-8ff2-a4a657770227 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.906942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 388/768] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:49.906942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39831676-8ab3-4434-8ff2-a4a657770227 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.910087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 381/769] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:49.910812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.014961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39831676-8ab3-4434-8ff2-a4a657770227 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.017905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a745d-304b-457b-8b47-7057b360b2f2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.046637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.053245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b326900-50ee-40c9-915b-74c95c08caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.053872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b326900-50ee-40c9-915b-74c95c08caa2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.054153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b326900-50ee-40c9-915b-74c95c08caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.054529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b326900-50ee-40c9-915b-74c95c08caa2 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.054791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.055124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b326900-50ee-40c9-915b-74c95c08caa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.055473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.056401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.056686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.062168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b326900-50ee-40c9-915b-74c95c08caa2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.062783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 389/770] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.064151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3c8de-8824-4d21-adf3-7f0755b109dd None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.064388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3c8de-8824-4d21-adf3-7f0755b109dd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.064667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3c8de-8824-4d21-adf3-7f0755b109dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.068771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.070307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:50.084922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:50.085897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:50.086891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 382/771] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:50.087913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.101259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2fa3c8de-8824-4d21-adf3-7f0755b109dd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:50.102886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2fa3c8de-8824-4d21-adf3-7f0755b109dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:50.108532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.219476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 390/772] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:50.219476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3c8de-8824-4d21-adf3-7f0755b109dd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.233748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d194b-8dae-4764-ae5e-f8449d91df72 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.276431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.277176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.277460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.277736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/9072775865604c73ae8361a047945464/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.279757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=9072775865604c73ae8361a047945464, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.281205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.346254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3c8de-8824-4d21-adf3-7f0755b109dd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.366484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 383/773] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:50 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/9072775865604c73ae8361a047945464/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:50.367445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.401153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.448850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.449641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.449972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.450730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.451009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:50.451355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.452915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.487648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.497253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 391/774] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.508257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.516702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b6327-8828-4b34-b2c5-363892925500 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.563787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.568924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.568924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.568924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] PATH_INFO: `/v3/services/network` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.568924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.569216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:50.585504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:50.586561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:50.587525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:50.588255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 384/775] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:50.596847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.706862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e28d3d-5fe6-411b-90ad-3fc2ee54ac13 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.756742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a7788-2c73-4fe6-ab17-3ecc44951910 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.766781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.767503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.767784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.768057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.770121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=a75c1093580244409a9074cc0d277af1) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:50.770446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.771926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.784208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 392/776] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/role_assignments?user.id=a75c1093580244409a9074cc0d277af1&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:50.820080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:50.832785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.833806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.834262 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.834747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.835663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] RBAC: Adding query filter params (name=network) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:50.836223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.838413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.854507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 385/777] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:50.860908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.956288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:51.016009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578a20a-d0ed-4495-8ebb-cdd43ed0b026 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:51.061335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:51.061976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.062254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.062532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.063221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] RBAC: Adding query filter params (type=network) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:51.063536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:51.064959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:51.076569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 393/778] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:51.082096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:51.229852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c7896-8170-4e63-8619-8f5091b85439 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:51.278469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666b4ab-727c-4996-a786-435b0ec99450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:51.279116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666b4ab-727c-4996-a786-435b0ec99450 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.279430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666b4ab-727c-4996-a786-435b0ec99450 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.279715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666b4ab-727c-4996-a786-435b0ec99450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.280487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666b4ab-727c-4996-a786-435b0ec99450 None admin] RBAC: Adding query filter params (interface=public, service_id=32f7bb6fb17345dab4c3f5343268a427, region_id=RegionOne) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:51.280804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666b4ab-727c-4996-a786-435b0ec99450 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:51.282192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666b4ab-727c-4996-a786-435b0ec99450 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:51.293966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 386/779] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:01:51 2022] GET /identity/v3/endpoints?service_id=32f7bb6fb17345dab4c3f5343268a427&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:51.299099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:51.485528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e86791-8854-4ae1-b65c-b8d7796f8bef None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:51.501308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:51.523546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79b7b2-4ab3-4a99-bb06-5055cba27225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:51.524169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b7b2-4ab3-4a99-bb06-5055cba27225 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.524509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b7b2-4ab3-4a99-bb06-5055cba27225 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.525351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b7b2-4ab3-4a99-bb06-5055cba27225 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.526008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79b7b2-4ab3-4a99-bb06-5055cba27225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:51.527433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79b7b2-4ab3-4a99-bb06-5055cba27225 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:51.533073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 394/780] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:01:51 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:51.902039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95c9f-910d-4380-afbe-5fbfd9680ec5 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.902756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95c9f-910d-4380-afbe-5fbfd9680ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.903285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95c9f-910d-4380-afbe-5fbfd9680ec5 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.905195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 387/781] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:51.910802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddbee-7a2d-46fd-b29f-66fb79b21dfd None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:51.911403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddbee-7a2d-46fd-b29f-66fb79b21dfd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:51.911890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddbee-7a2d-46fd-b29f-66fb79b21dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:51.931533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-199ddbee-7a2d-46fd-b29f-66fb79b21dfd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:51.933052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-199ddbee-7a2d-46fd-b29f-66fb79b21dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:52.017150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 395/782] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:52.057549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd95c9f-910d-4380-afbe-5fbfd9680ec5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.145905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.200963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd95c9f-910d-4380-afbe-5fbfd9680ec5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.254729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791487de-52dc-4c8b-b02d-d6dd11f590da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.255416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791487de-52dc-4c8b-b02d-d6dd11f590da None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.255705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791487de-52dc-4c8b-b02d-d6dd11f590da None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.255981 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791487de-52dc-4c8b-b02d-d6dd11f590da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.269378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791487de-52dc-4c8b-b02d-d6dd11f590da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-791487de-52dc-4c8b-b02d-d6dd11f590da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:52.276338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:52.277273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:52.278097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 388/783] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:52.288307 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddbee-7a2d-46fd-b29f-66fb79b21dfd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.392833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fc171-bec9-482c-a3c6-f407ad620ce9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.393157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fc171-bec9-482c-a3c6-f407ad620ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.393396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fc171-bec9-482c-a3c6-f407ad620ce9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.394955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 389/784] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:52.400573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e169a-06e1-4f22-b505-1a7754f90709 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.400857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e169a-06e1-4f22-b505-1a7754f90709 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.401135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e169a-06e1-4f22-b505-1a7754f90709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.421030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddbee-7a2d-46fd-b29f-66fb79b21dfd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.435573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.436533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dc2e169a-06e1-4f22-b505-1a7754f90709 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:52.437765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dc2e169a-06e1-4f22-b505-1a7754f90709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:52.465113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.465659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.465941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.466214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.466929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:52.467255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.468625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.477306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 396/785] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:52.485897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.538683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 390/786] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:01:52.597274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e169a-06e1-4f22-b505-1a7754f90709 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.655685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.667516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49049f-b826-4a1b-8f4b-2da3035beca4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.718941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e169a-06e1-4f22-b505-1a7754f90709 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.737125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.737785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.738067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.738345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.761687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.763073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.763230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.764025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.764413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.764795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.765544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.767025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:52.776930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:52.778102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:01:52.779151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:52.785988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 391/787] 10.222.0.49 () {56 vars in 1129 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:52.787324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:52.794537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:52.795678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:52.797514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:52.797514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:52.797514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:52.797514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:52.808190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 397/788] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:52.813902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.858683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.900342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc03954-f61f-4b05-a060-af8119a46315 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.915953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.945391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.946040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.946348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.946629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.947463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:52.947774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.949360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.963597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 392/789] 10.222.0.49 () {56 vars in 1124 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:52.969736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.970180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cc6cd-257d-4677-a86f-41172ae4f0df None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.017859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.018492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.018724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.018941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.019665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:53.019967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.021339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.043283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 398/790] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:53.051865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.172266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86685ff5-0f3a-43a7-86e9-ea0ffc43b010 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.198826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.209448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf2316-14e1-422f-bb4e-78a125a1ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.210070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2316-14e1-422f-bb4e-78a125a1ef06 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.210358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2316-14e1-422f-bb4e-78a125a1ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.210631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2316-14e1-422f-bb4e-78a125a1ef06 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.211337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2316-14e1-422f-bb4e-78a125a1ef06 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:53.211645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2316-14e1-422f-bb4e-78a125a1ef06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.213247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2316-14e1-422f-bb4e-78a125a1ef06 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.218359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ac8c5-ec94-4e2e-bd1f-1cb46d68ad44 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.229483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 393/791] 10.222.0.49 () {56 vars in 1124 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:53.229483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330644e-afb1-4f7f-95c0-f1179399f5bd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.229483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330644e-afb1-4f7f-95c0-f1179399f5bd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.229483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330644e-afb1-4f7f-95c0-f1179399f5bd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.229483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 394/792] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:53.229483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330644e-afb1-4f7f-95c0-f1179399f5bd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.268879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631b590-bf7a-44a8-b08d-807306ece501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.270193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631b590-bf7a-44a8-b08d-807306ece501 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.270803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631b590-bf7a-44a8-b08d-807306ece501 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.271353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631b590-bf7a-44a8-b08d-807306ece501 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.280753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631b590-bf7a-44a8-b08d-807306ece501 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.282975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631b590-bf7a-44a8-b08d-807306ece501 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.287087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4631b590-bf7a-44a8-b08d-807306ece501 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:53.301571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:53.302545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:53.303498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 399/793] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:53.304107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baeae8-2dde-4b9c-a71e-27e55de44319 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.305016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baeae8-2dde-4b9c-a71e-27e55de44319 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.305016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baeae8-2dde-4b9c-a71e-27e55de44319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.322749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-93baeae8-2dde-4b9c-a71e-27e55de44319 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:53.323979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-93baeae8-2dde-4b9c-a71e-27e55de44319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:53.357131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330644e-afb1-4f7f-95c0-f1179399f5bd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.393420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 400/794] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:53.393743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba2993-c6e5-46e9-88dc-065dedd190d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.394654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2993-c6e5-46e9-88dc-065dedd190d0 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.395114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2993-c6e5-46e9-88dc-065dedd190d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.395346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432a5d2-c169-4faa-87ca-65d10d814751 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.395711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba2993-c6e5-46e9-88dc-065dedd190d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.395824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432a5d2-c169-4faa-87ca-65d10d814751 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.396156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432a5d2-c169-4faa-87ca-65d10d814751 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.396822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba2993-c6e5-46e9-88dc-065dedd190d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.397832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 401/795] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:53.398632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432a5d2-c169-4faa-87ca-65d10d814751 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.399048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba2993-c6e5-46e9-88dc-065dedd190d0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.477694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.481574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 395/796] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:53.483046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef6a8f-6b14-4244-a973-5b3b9fda4c56 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.483510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef6a8f-6b14-4244-a973-5b3b9fda4c56 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.483918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef6a8f-6b14-4244-a973-5b3b9fda4c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.506657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0aef6a8f-6b14-4244-a973-5b3b9fda4c56 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:53.508445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0aef6a8f-6b14-4244-a973-5b3b9fda4c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:53.549917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432a5d2-c169-4faa-87ca-65d10d814751 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.590094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.590933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.591335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.591689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.592660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:53.593102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.594699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.610536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 396/797] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:53.611813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef6a8f-6b14-4244-a973-5b3b9fda4c56 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.631718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 402/798] 10.222.0.49 () {56 vars in 1135 bytes} [Tue Aug 2 10:01:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:53.632862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.651389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.741996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef6a8f-6b14-4244-a973-5b3b9fda4c56 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.774507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c14cfa-c975-425d-88ae-190ed9d62ec1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.784582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.785239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.785523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.785798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.797260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:53.802185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:53.803119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:53.804412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:53.807417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 397/799] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:53.808154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.815448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.829364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.835155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.835155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.835155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.845768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.848018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:53.867108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:53.869106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:53.870084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:53.870084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:53.870084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:53.870084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:53.875379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 403/800] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:53.875379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.921155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.934876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcede5e6-3dca-4ea8-b402-6531da343b4a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.973502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.974248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.974624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.974965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.989685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:53.996650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:53.997569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:53.998481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:54.003059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 398/801] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:54.004278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.019499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78819ddc-bf34-4f4a-b773-7bb7ecddcdaf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.067554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.068227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.068621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.068907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.069663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:54.069969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.071496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.080601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 404/802] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.081409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.124289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d6767-a450-4b99-b85b-03455379746c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.127501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.160153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4eeb3e-b176-4922-9c93-c4f36d5e7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.160773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eeb3e-b176-4922-9c93-c4f36d5e7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.161058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eeb3e-b176-4922-9c93-c4f36d5e7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.161337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eeb3e-b176-4922-9c93-c4f36d5e7ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.162030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4eeb3e-b176-4922-9c93-c4f36d5e7ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:54.162340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4eeb3e-b176-4922-9c93-c4f36d5e7ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.163751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4eeb3e-b176-4922-9c93-c4f36d5e7ec8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.180922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 399/803] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.180922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a97376-581f-4c10-9e16-f4a2cf505e67 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.180922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a97376-581f-4c10-9e16-f4a2cf505e67 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.180922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a97376-581f-4c10-9e16-f4a2cf505e67 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.183730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 400/804] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:54.184423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a97376-581f-4c10-9e16-f4a2cf505e67 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.195168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.198374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd286-765c-499d-b498-870e0c7454a4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.235482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.236105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.236387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.236658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.237363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:54.237667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.239044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.247716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 405/805] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.252826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.293333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a97376-581f-4c10-9e16-f4a2cf505e67 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.328372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d06f2-38ee-4f0d-bb4b-378b97b31e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.329063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d06f2-38ee-4f0d-bb4b-378b97b31e49 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.329358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d06f2-38ee-4f0d-bb4b-378b97b31e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.329638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d06f2-38ee-4f0d-bb4b-378b97b31e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.339959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d06f2-38ee-4f0d-bb4b-378b97b31e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.341426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d06f2-38ee-4f0d-bb4b-378b97b31e49 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-159d06f2-38ee-4f0d-bb4b-378b97b31e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:54.356863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:54.357761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:54.358699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:54.358699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:54.358699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:54.358699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:54.359739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1efbd-86b6-4d38-95b8-77dfc4fd6632 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.365257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 401/806] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:54.365257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0413c1-f42e-434a-9406-3024b8d06459 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.365257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0413c1-f42e-434a-9406-3024b8d06459 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.365257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0413c1-f42e-434a-9406-3024b8d06459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.366146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.382473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-eb0413c1-f42e-434a-9406-3024b8d06459 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:54.383722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-eb0413c1-f42e-434a-9406-3024b8d06459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.407178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.408102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.408797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.409089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.418477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.420122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:54.435429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:54.436317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:54.437382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:54.437811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 406/807] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:54.438344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.444217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.459240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 402/808] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:54.459240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0413c1-f42e-434a-9406-3024b8d06459 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.566578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271515dd-5195-4a3d-b657-42a28c5f3327 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.577179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0413c1-f42e-434a-9406-3024b8d06459 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.602283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.603600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.603898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.604188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.614824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.615493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.615784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.616066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.616165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.616765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:54.617077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.617606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.618479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.635226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 403/809] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.635226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:54.635466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:54.636290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:54.637164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:54.637164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:54.637164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:54.637164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:54.637164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 407/810] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:54.637164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.654729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.698643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.753164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c9293-cc14-4250-98dd-db3612a94086 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.754016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7efb-4feb-487c-b6dc-24e2101b6bc0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.788930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.789472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.789758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.790036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.791495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:54.791808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.793254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.795077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.796032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.796320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.796599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] PATH_INFO: `/v3/services/network` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.797238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.798732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.810929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 404/811] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.810929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:54.811507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:54.812442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:54.813265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 408/812] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:54.814054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.895420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.919802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf49451-6dba-47de-9a3b-2903c55515eb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.922754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e2796-3775-431f-a99a-5352cb2f21fb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.934434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.957260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.957981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.958087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.958304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.958537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.958708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.958990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.959243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:54.959354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.959556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.961255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.967511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.969257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.975478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 409/813] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:54.976627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:54.985875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:54.986947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 405/814] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:54.988000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.097050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d037-b72a-47e7-acdf-712e5cd8ea32 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.097875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f954-df27-4d81-9dd3-58268af0d2b9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.133293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.134680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.134961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.135257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/66f7db7eea94472c82cda44c569d9550/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.136509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.137136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.137368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=66f7db7eea94472c82cda44c569d9550, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.137470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.137685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.138362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:55.138673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.138842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.140245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.150326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 410/815] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.150326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.150326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.245175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768d9f6-4a33-42ba-a847-0488c4ce5ba2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.260232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 406/816] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:54 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/66f7db7eea94472c82cda44c569d9550/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:55.261079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.296297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.297873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.298369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.298855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.310682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.310921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.312738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:55.329150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:55.330176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:01:55.331115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:55.336347 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 411/817] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:55.337855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.420876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.437946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cef741-372a-491f-a8d7-973d37ee61b9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.471350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd81f1-0375-4f54-99df-106b902977e8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.480425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.481511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.482001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.482595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.483656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:55.484169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.485986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.506175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 407/818] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.511211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.511211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.511211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.511211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.511597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] RBAC: Adding query filter params (type=network) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:55.511597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.511597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.512246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.519962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 412/819] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.520895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.608601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.624525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cc617-13b9-4782-aa82-857a9711a1a5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.629323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0de757-b407-4e3e-b64f-9cca1e980f19 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.656426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.666475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbca9e-477c-4436-8df3-bb5c86da697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.667295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbca9e-477c-4436-8df3-bb5c86da697c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.667679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbca9e-477c-4436-8df3-bb5c86da697c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.668055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbca9e-477c-4436-8df3-bb5c86da697c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.669091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.669715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.669997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.670267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.671165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:55.671430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.672866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.696770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 408/820] 10.222.0.49 () {56 vars in 1149 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.696770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.751374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbca9e-477c-4436-8df3-bb5c86da697c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.753483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbca9e-477c-4436-8df3-bb5c86da697c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.800721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21fb33-1cd5-4579-977f-52a38180fef0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.836987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.837637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.837918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.838195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.838899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.840303 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.863148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 413/821] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:01:55.864790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a795a-dcff-4ea6-9b9e-4c311a815d41 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.865079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a795a-dcff-4ea6-9b9e-4c311a815d41 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.865358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a795a-dcff-4ea6-9b9e-4c311a815d41 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.868055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 414/822] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:55.868055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a795a-dcff-4ea6-9b9e-4c311a815d41 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.894600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.955994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 409/823] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/endpoints => generated 337 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:55.957140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.960176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.983974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a795a-dcff-4ea6-9b9e-4c311a815d41 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.035761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8068ab6-dea1-40ac-941d-4d1bdc691243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.036107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8068ab6-dea1-40ac-941d-4d1bdc691243 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.036580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8068ab6-dea1-40ac-941d-4d1bdc691243 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.036697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8068ab6-dea1-40ac-941d-4d1bdc691243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.048617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8068ab6-dea1-40ac-941d-4d1bdc691243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.050192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8068ab6-dea1-40ac-941d-4d1bdc691243 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e8068ab6-dea1-40ac-941d-4d1bdc691243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:56.067453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:56.073786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 415/824] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90026-64a4-46d3-a044-6723b6457cee None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90026-64a4-46d3-a044-6723b6457cee None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.074906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90026-64a4-46d3-a044-6723b6457cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.091816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ecf90026-64a4-46d3-a044-6723b6457cee None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:56.093141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ecf90026-64a4-46d3-a044-6723b6457cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:56.110674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d63ed-be29-456d-af79-440cd076b9f9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.159627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.160290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.160574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.160848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.173696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.176405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.179318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 416/825] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:56.180214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf90026-64a4-46d3-a044-6723b6457cee None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.184246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:56.195021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:56.198223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:56.199110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:56.199110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:56.199110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:56.199110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:56.199986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 410/826] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:56.200211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.271436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.308751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf90026-64a4-46d3-a044-6723b6457cee None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.318518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a989-01ef-48df-8668-1a4387c803a5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.345785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.346407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.346689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.346962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.347687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:56.347986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.349375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.360501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.361152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.366234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.366395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 417/827] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.366491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.366577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.366662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.366662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:56.376525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:56.377731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:01:56.378772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:56.382084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 411/828] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:56.382985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.438861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.480123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686688bb-9ba3-4794-8143-33fd144c1bdb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.489391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3ee2a-cfea-4852-851c-aa2a40d08fa5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.513273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.521669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.522281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.522517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.522793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.523530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:56.523856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.525778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.531717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.532483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.532661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.532907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.539771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.541249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.543763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 418/829] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.544616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:56.557138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:56.558266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 412/830] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:56.559256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.668752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58bbb9-4830-4d6a-ad7b-2d735159dd82 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.669411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf3a99-a87e-4fbf-83ca-a757fffe8060 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.694099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.704091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.713712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.713712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.713712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.713934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.713934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.713934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.713934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.714538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:56.714538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.714538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.714745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.714818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 419/831] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.715062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.718408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:56.730241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:56.731455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 413/832] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:56.732724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.775787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.826252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28160fa-693d-4b78-a059-da3ab5a23c76 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.841106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce607-b0c6-4a23-8b22-927d1ee2acaa None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.865925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.866580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.866863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.867137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.867920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:56.868232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.870480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.879106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.879757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.880046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.880323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.880995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:56.881300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.882660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.889527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 414/833] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.889527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.889961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 420/834] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:56.896762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.953799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.017219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6325d70-627b-4740-a541-1f329552d5c1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.030242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341392d9-9f3e-4f3d-9824-0ad22e762b00 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.031973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.060106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb252214-ef38-4b9d-b232-39aa39c536d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.061034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252214-ef38-4b9d-b232-39aa39c536d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.061466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252214-ef38-4b9d-b232-39aa39c536d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.061891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252214-ef38-4b9d-b232-39aa39c536d8 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/a75c1093580244409a9074cc0d277af1/roles/b0563004ff664008b6db05941cf9c9b9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.068689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.069313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.069596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.069872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.070625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:57.070934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.072508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.083258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb252214-ef38-4b9d-b232-39aa39c536d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=a75c1093580244409a9074cc0d277af1, role_id=b0563004ff664008b6db05941cf9c9b9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.084715 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb252214-ef38-4b9d-b232-39aa39c536d8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.086962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 415/835] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:57.094050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.201859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 421/836] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:01:56 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/a75c1093580244409a9074cc0d277af1/roles/b0563004ff664008b6db05941cf9c9b9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:57.211077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5814ae-2a32-45e2-8e29-5e5680515668 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.251036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b9caf-838d-48a2-80bf-39ca7c27e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.251828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b9caf-838d-48a2-80bf-39ca7c27e221 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.252179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b9caf-838d-48a2-80bf-39ca7c27e221 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.252515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b9caf-838d-48a2-80bf-39ca7c27e221 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.253425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b9caf-838d-48a2-80bf-39ca7c27e221 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=9072775865604c73ae8361a047945464) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:57.253807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b9caf-838d-48a2-80bf-39ca7c27e221 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.255355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b9caf-838d-48a2-80bf-39ca7c27e221 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.263322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 416/837] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/role_assignments?user.id=9072775865604c73ae8361a047945464&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:57.417864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f14c9-7511-4314-97ea-2cf32961bd78 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.418055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f14c9-7511-4314-97ea-2cf32961bd78 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.418348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f14c9-7511-4314-97ea-2cf32961bd78 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.420108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 422/838] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:57.425720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700e55-842c-413a-b889-e321cabbb60f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.426218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700e55-842c-413a-b889-e321cabbb60f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.426299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700e55-842c-413a-b889-e321cabbb60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.445394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5e700e55-842c-413a-b889-e321cabbb60f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:57.447125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5e700e55-842c-413a-b889-e321cabbb60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:57.470064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.522778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 417/839] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:57.550554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f14c9-7511-4314-97ea-2cf32961bd78 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.661603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.666989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f14c9-7511-4314-97ea-2cf32961bd78 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.703366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.704962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.705461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.705945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.717603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:57.722954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:57.723876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:01:57.724708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:57.727133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 423/840] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:57.734465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700e55-842c-413a-b889-e321cabbb60f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.842178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700e55-842c-413a-b889-e321cabbb60f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.878464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.879118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.879457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.879739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.880461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:57.880772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.882158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.891691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 418/841] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:57.897250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.004536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479bbeb-8561-48cc-afe8-af575ac62698 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.042292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.043366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.043857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.044351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.057887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.059795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:58.075976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:58.077028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:58.078020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:58.078020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:58.078020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:58.078020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:58.080527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 424/842] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:58.085834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.111615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.194640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67a115-153c-4da4-9399-c6db20da3385 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.229808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.230433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.230723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.231007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.231729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:58.232041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.233429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.246377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 419/843] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:58.252028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.265228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.355000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e179c93-65d0-48b9-ac87-377316fb5938 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.393482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.394483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.394997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.395508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.403843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.405566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:01:58.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:58.418705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:01:58.419686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:01:58.422248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 425/844] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:58.427725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.519811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.545932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b5d95-587f-40ab-8c7d-fc5ca4ce5874 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.590825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.592679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.592679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.592679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.593592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:58.593786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.595349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.616261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 420/845] 10.222.0.49 () {56 vars in 1135 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:58.624312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.723546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.749052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3820df-228c-4f06-8471-79b7acf5ecda None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.785809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e1a8b-cfdb-4e24-9d01-d1002afd7991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.786507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e1a8b-cfdb-4e24-9d01-d1002afd7991 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.786895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e1a8b-cfdb-4e24-9d01-d1002afd7991 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.787308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e1a8b-cfdb-4e24-9d01-d1002afd7991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.861041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e1a8b-cfdb-4e24-9d01-d1002afd7991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.862624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e1a8b-cfdb-4e24-9d01-d1002afd7991 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.936681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.972786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 426/846] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:01:58.984776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.095584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2ca55-ce64-4c26-af1c-aa39bfe3e900 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.139756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.140830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.141372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.141851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.145257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.149072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccaae-4222-493a-930e-7d63aa1f9e81 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.149370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccaae-4222-493a-930e-7d63aa1f9e81 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.149647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccaae-4222-493a-930e-7d63aa1f9e81 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.154848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.154975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 427/847] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:59.158947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8547-f2c6-42c3-aa7b-6681aab54a48 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.158947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8547-f2c6-42c3-aa7b-6681aab54a48 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.159130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.159246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8547-f2c6-42c3-aa7b-6681aab54a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.182628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-768c8547-f2c6-42c3-aa7b-6681aab54a48 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:59.188643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-768c8547-f2c6-42c3-aa7b-6681aab54a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:01:59.192751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:59.193651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:59.194662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:01:59.194662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:01:59.194662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:01:59.194662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:59.203529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 421/848] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:59.204761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.291469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 428/849] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:59.338582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c8547-f2c6-42c3-aa7b-6681aab54a48 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.389811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6238a0-0afa-466d-a2ba-23344dd9bba8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.433965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.434692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.434879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.435158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.435877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:59.436181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.437671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.451321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 422/850] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:59.456830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.464250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.467255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c8547-f2c6-42c3-aa7b-6681aab54a48 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.507924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bb12d-4f2c-4604-9427-f6b502783ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.508669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bb12d-4f2c-4604-9427-f6b502783ba0 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.509026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bb12d-4f2c-4604-9427-f6b502783ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.509878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bb12d-4f2c-4604-9427-f6b502783ba0 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.510659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bb12d-4f2c-4604-9427-f6b502783ba0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.512180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bb12d-4f2c-4604-9427-f6b502783ba0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.520456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.572673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ef2c-b79d-4bf0-9b93-b636ff0548b9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.590776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 429/851] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:01:59 2022] POST /identity/v3/services => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:59.593302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9d5a-e8df-4bd6-9a34-f80725533d12 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.593769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9d5a-e8df-4bd6-9a34-f80725533d12 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.594135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9d5a-e8df-4bd6-9a34-f80725533d12 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.595640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 430/852] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:59.601102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5b999-7038-47cc-9deb-29e858292a3f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.601494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5b999-7038-47cc-9deb-29e858292a3f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.601770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5b999-7038-47cc-9deb-29e858292a3f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.607249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 431/853] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:01:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:01:59.607249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cf78-2b6a-42ca-b742-0b7345fda1c9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.607249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cf78-2b6a-42ca-b742-0b7345fda1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.607249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cf78-2b6a-42ca-b742-0b7345fda1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.622197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f01b2-a286-4b91-8d15-7c43a15a8658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.623335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f01b2-a286-4b91-8d15-7c43a15a8658 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.623770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f01b2-a286-4b91-8d15-7c43a15a8658 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.624246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f01b2-a286-4b91-8d15-7c43a15a8658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.625343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b3f0cf78-2b6a-42ca-b742-0b7345fda1c9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:59.626592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b3f0cf78-2b6a-42ca-b742-0b7345fda1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:59.632842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f01b2-a286-4b91-8d15-7c43a15a8658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.635102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f01b2-a286-4b91-8d15-7c43a15a8658 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-079f01b2-a286-4b91-8d15-7c43a15a8658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:01:59.647940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:59.650019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:01:59.651627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:59.653652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 423/854] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:01:59.655147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62c8d8-7244-4c70-a9d2-7aabf92d18fd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.655467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62c8d8-7244-4c70-a9d2-7aabf92d18fd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.655751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62c8d8-7244-4c70-a9d2-7aabf92d18fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.674675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-af62c8d8-7244-4c70-a9d2-7aabf92d18fd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:59.676074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-af62c8d8-7244-4c70-a9d2-7aabf92d18fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:59.715463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 432/855] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:59.715463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0cf78-2b6a-42ca-b742-0b7345fda1c9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.759625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 424/856] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:01:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:01:59.760479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62c8d8-7244-4c70-a9d2-7aabf92d18fd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.800148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.830118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.842394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0cf78-2b6a-42ca-b742-0b7345fda1c9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.881224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.881467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62c8d8-7244-4c70-a9d2-7aabf92d18fd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.881852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.882138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.882418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.883384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:59.884706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.886161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.905432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 433/857] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:59.905432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.920851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.921501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.921784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.922060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.932709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:01:59.937695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:01:59.938608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 425/858] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:01:59.940329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.022464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5d6ee-929f-435f-895b-c0b80af8bf4e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.036906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.044416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482fe3-eaa4-446f-9b56-8b9e9cee829b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.060360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.061106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.061466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.061821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.062526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.064069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:00.073766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:00.074717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:00.075598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:00.079996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 434/859] 10.222.0.49 () {56 vars in 1132 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:00.080836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.082511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.083218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.083505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.083780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.084574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=66f7db7eea94472c82cda44c569d9550) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:00.084889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.086337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.093353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 426/860] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/role_assignments?user.id=66f7db7eea94472c82cda44c569d9550&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:00.093527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.104533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.229781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a69af-ce79-4b37-b596-dbc5a7bf0f19 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.230715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a5e09-486d-418c-b05d-e910aefbdbed None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.277220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.277860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.278142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.278255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.278445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.278867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.279137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:00.279327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.279472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.279561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.280144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:00.280450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.280934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.281806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.288976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 427/861] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:00.292488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 435/862] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:00.296232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.297861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.324941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.398149 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.411488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d4d9-745a-4636-9e85-2326d9023946 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.428601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd1f15-ec9a-4e6b-9728-2e582d9c7cdd None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.448200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.448835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.449107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.449380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.450056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:00.450408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.451886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.456983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 428/863] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:00.471852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.472528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.472812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.473092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.487006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.488444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:00.505776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:00.506730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:00.508358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:00.508358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:00.508358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:00.508358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:00.508358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 436/864] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:00.513612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.627587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.632808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874081b-2415-4bf4-baa0-3aaf7905674f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.671329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.672343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.672671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.672951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.673643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:00.674006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.675411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.688406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 429/865] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:00.693862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.805110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e718475-0f5f-40f3-93ea-add79b7b6b3c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:00.833264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:00.841820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:00.842482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:00.842780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:00.843425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:00.853764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:00.853935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:00.866054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:00.867044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:00.868030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 437/866] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:00.873296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:00.977433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60c90-7eda-4f48-ac05-7209c9b65310 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.012576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.013196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.013477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.013750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.014535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:01.014839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.016291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.035343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 430/867] 10.222.0.49 () {56 vars in 1149 bytes} [Tue Aug 2 10:02:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:01.041118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.146373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133671fc-b948-45c4-8368-e7f699416e62 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.183116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.183773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.184060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.184341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.247893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.260904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.264605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.367216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 438/868] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:01.373241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.382996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.486094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeab32-9d23-4159-afce-3abee8edb883 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.523215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.524233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.524714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.525219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.540069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.541645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:01.557069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:01.557954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:01.558902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:01.558902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:01.558902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:01.558902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:01.558902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 431/869] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:01.565241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.656026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.672221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7132d-a2bd-48c9-bc39-98050abf3087 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.709013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63489c-708a-4a0f-bff4-072654159353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.709643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63489c-708a-4a0f-bff4-072654159353 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.709926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63489c-708a-4a0f-bff4-072654159353 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.710201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63489c-708a-4a0f-bff4-072654159353 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.710896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63489c-708a-4a0f-bff4-072654159353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:01.711225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63489c-708a-4a0f-bff4-072654159353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.712653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63489c-708a-4a0f-bff4-072654159353 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.724837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 439/870] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:01.730855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.737434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a24e93-f8cd-4c62-97e9-e876d5da3040 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.737681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a24e93-f8cd-4c62-97e9-e876d5da3040 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.737937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a24e93-f8cd-4c62-97e9-e876d5da3040 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.739560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 440/871] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:01.744925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a0b0c-cfb1-4407-8090-0afaa5519038 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.745725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a0b0c-cfb1-4407-8090-0afaa5519038 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.746230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a0b0c-cfb1-4407-8090-0afaa5519038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.767153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-864a0b0c-cfb1-4407-8090-0afaa5519038 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:01.768720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-864a0b0c-cfb1-4407-8090-0afaa5519038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:01.807165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:01.846243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 441/872] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:01.862773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e4e33-f864-4d13-bede-7982053dd918 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:01.870920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a0b0c-cfb1-4407-8090-0afaa5519038 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.901798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:01.902787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:01.903345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:01.903846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:01.911049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:01.912760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:01.925615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:01.926534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:01.927482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:01.929863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 432/873] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:01.936067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:01.972387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.001686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a0b0c-cfb1-4407-8090-0afaa5519038 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.045624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590736c-eec0-4ee1-9c77-64a2e943d541 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.046702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.047383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.047665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.047941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.048734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.050973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:02.060763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:02.061783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:02.062618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:02.070332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 442/874] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:02.071538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.072684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.089619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.090498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.090785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.091056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.092335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.092807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.094554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.115272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 433/875] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:02.118053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.197927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe457a-a5f0-4035-9029-9ec7625af278 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.239720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9f64-0f19-4172-ba3e-f2f02268f05f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.246628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.247532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.248015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.249033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.249974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.250924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.253316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.266383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 443/876] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.266691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.277621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.285967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ec3df-239a-4a05-9c96-a46c233b5297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.287257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ec3df-239a-4a05-9c96-a46c233b5297 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.287837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ec3df-239a-4a05-9c96-a46c233b5297 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.295358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ec3df-239a-4a05-9c96-a46c233b5297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.295358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ec3df-239a-4a05-9c96-a46c233b5297 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=a75c1093580244409a9074cc0d277af1) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.295358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ec3df-239a-4a05-9c96-a46c233b5297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.295358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ec3df-239a-4a05-9c96-a46c233b5297 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.307661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 434/877] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/role_assignments?user.id=a75c1093580244409a9074cc0d277af1&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:02.315517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c30e3-5879-4cff-bc06-a0658e1ad8a9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.315865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c30e3-5879-4cff-bc06-a0658e1ad8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.316087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c30e3-5879-4cff-bc06-a0658e1ad8a9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.318085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 435/878] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:02.323898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997668b-8bb7-4421-a681-5669abc8a355 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.324197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997668b-8bb7-4421-a681-5669abc8a355 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.324490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997668b-8bb7-4421-a681-5669abc8a355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.347064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4997668b-8bb7-4421-a681-5669abc8a355 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:02.348342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4997668b-8bb7-4421-a681-5669abc8a355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:02.389551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efae7ae-e030-4c50-9398-260be5f0c62d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.399553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.433482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.434217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.434708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.435141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.436011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.436326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.437807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.444016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 444/879] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.449990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.450204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 436/880] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:02.477323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997668b-8bb7-4421-a681-5669abc8a355 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.548236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.597911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33bf97-7912-42ed-815f-2a4d484421c0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.617009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997668b-8bb7-4421-a681-5669abc8a355 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.633281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db37bf-36ef-4e3b-9386-43a1439b111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.633960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db37bf-36ef-4e3b-9386-43a1439b111b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.634193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db37bf-36ef-4e3b-9386-43a1439b111b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.634988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db37bf-36ef-4e3b-9386-43a1439b111b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.635801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db37bf-36ef-4e3b-9386-43a1439b111b None admin] RBAC: Adding query filter params (interface=public, service_id=99294b89e70e49c18202886aaeeb93d4, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.636114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db37bf-36ef-4e3b-9386-43a1439b111b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.637477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db37bf-36ef-4e3b-9386-43a1439b111b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.642925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 445/881] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/endpoints?service_id=99294b89e70e49c18202886aaeeb93d4&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.647225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b351e-799b-4772-9d81-010c9e0b9ead None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.647225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b351e-799b-4772-9d81-010c9e0b9ead None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.647225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b351e-799b-4772-9d81-010c9e0b9ead None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.647225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 446/882] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:02.650932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b351e-799b-4772-9d81-010c9e0b9ead None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.660715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c534cd-fbd8-4a4b-9150-4883f304db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.661367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c534cd-fbd8-4a4b-9150-4883f304db6b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.661648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c534cd-fbd8-4a4b-9150-4883f304db6b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.661923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c534cd-fbd8-4a4b-9150-4883f304db6b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.662577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c534cd-fbd8-4a4b-9150-4883f304db6b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.663968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c534cd-fbd8-4a4b-9150-4883f304db6b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-02c534cd-fbd8-4a4b-9150-4883f304db6b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:02.676025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:02.677058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:02.677899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 437/883] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:02.678712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657b0f-eff5-48c9-b759-97e1b907fb76 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.678712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657b0f-eff5-48c9-b759-97e1b907fb76 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.679631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5657b0f-eff5-48c9-b759-97e1b907fb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.698806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b5657b0f-eff5-48c9-b759-97e1b907fb76 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:02.700132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b5657b0f-eff5-48c9-b759-97e1b907fb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:02.730888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.772286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b351e-799b-4772-9d81-010c9e0b9ead None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.774096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 438/884] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:02.776558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657b0f-eff5-48c9-b759-97e1b907fb76 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.811549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.812144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.812436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.812805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.813473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.814849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.816168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:02.820991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 447/885] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/services => generated 1011 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:02.822886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.925276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5657b0f-eff5-48c9-b759-97e1b907fb76 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.944707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0f79-e2e2-4d45-884f-c7a6a9b2a8f5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:02.967755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.968709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.969193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.969678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.970662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:02.971282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.972849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:02.979308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 439/886] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:02.985117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:02.990073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743338c1-3dcf-4bf5-9b99-921f02cc4ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:02.990718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743338c1-3dcf-4bf5-9b99-921f02cc4ba4 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:02.990997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743338c1-3dcf-4bf5-9b99-921f02cc4ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:02.991300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743338c1-3dcf-4bf5-9b99-921f02cc4ba4 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:02.992004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743338c1-3dcf-4bf5-9b99-921f02cc4ba4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:02.993391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743338c1-3dcf-4bf5-9b99-921f02cc4ba4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.019981 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:03.099063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 448/887] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:02 2022] POST /identity/v3/services => generated 269 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:03.099944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab1a13-0ea8-4160-9883-bfcf0bf1475e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:03.154398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c02b9-69a3-4281-af1a-5fce658c9093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:03.155533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c02b9-69a3-4281-af1a-5fce658c9093 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:03.155961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c02b9-69a3-4281-af1a-5fce658c9093 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:03.156472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c02b9-69a3-4281-af1a-5fce658c9093 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:03.157318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c02b9-69a3-4281-af1a-5fce658c9093 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:03.157883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c02b9-69a3-4281-af1a-5fce658c9093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:03.159583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c02b9-69a3-4281-af1a-5fce658c9093 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:03.166489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 440/888] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:04.500603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2996a-1491-4282-9b85-75ad60208c0b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:04.501648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2996a-1491-4282-9b85-75ad60208c0b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:04.502248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2996a-1491-4282-9b85-75ad60208c0b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:04.504125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 449/889] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:04.510810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb63c34-ccd7-4e6f-af27-e6ccb767389c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:04.511277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb63c34-ccd7-4e6f-af27-e6ccb767389c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:04.511579 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb63c34-ccd7-4e6f-af27-e6ccb767389c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:04.536319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ceb63c34-ccd7-4e6f-af27-e6ccb767389c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:04.537537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ceb63c34-ccd7-4e6f-af27-e6ccb767389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:04.608503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 441/890] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:04.631234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2996a-1491-4282-9b85-75ad60208c0b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:04.700085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:04.745609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2996a-1491-4282-9b85-75ad60208c0b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:04.780658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2052bd-63d4-4632-8ba8-61bcb1069a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:04.781240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2052bd-63d4-4632-8ba8-61bcb1069a2a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:04.781528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2052bd-63d4-4632-8ba8-61bcb1069a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:04.781809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2052bd-63d4-4632-8ba8-61bcb1069a2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:04.793871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2052bd-63d4-4632-8ba8-61bcb1069a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-be2052bd-63d4-4632-8ba8-61bcb1069a2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:04.798811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:04.799765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:04.800626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 450/891] 10.222.0.49 () {56 vars in 1120 bytes} [Tue Aug 2 10:02:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:04.808501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb63c34-ccd7-4e6f-af27-e6ccb767389c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:04.827727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:04.915684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb63c34-ccd7-4e6f-af27-e6ccb767389c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:04.945692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891eed9-d911-4072-97d1-a079d2138255 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:04.946009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891eed9-d911-4072-97d1-a079d2138255 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:04.946256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891eed9-d911-4072-97d1-a079d2138255 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:04.948425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 451/892] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:04.956642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39481209-48e5-452c-80b3-95029d861426 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:04.957138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39481209-48e5-452c-80b3-95029d861426 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:04.957521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39481209-48e5-452c-80b3-95029d861426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:04.958632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:04.959580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:04.960015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:04.960440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:04.961322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:04.961779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:04.963926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:04.974193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 442/893] 10.222.0.49 () {56 vars in 1115 bytes} [Tue Aug 2 10:02:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:04.980057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:04.982851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-39481209-48e5-452c-80b3-95029d861426 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:04.984072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-39481209-48e5-452c-80b3-95029d861426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:05.071699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 452/894] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:05.084054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.090270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39481209-48e5-452c-80b3-95029d861426 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.104812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645b04-396c-49a0-8485-c4a4c59ac8e1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.148967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.150122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024955d-ade1-4d63-9aae-653e6d80cc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.151719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024955d-ade1-4d63-9aae-653e6d80cc60 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.152048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024955d-ade1-4d63-9aae-653e6d80cc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.152329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024955d-ade1-4d63-9aae-653e6d80cc60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.167338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024955d-ade1-4d63-9aae-653e6d80cc60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.169501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024955d-ade1-4d63-9aae-653e6d80cc60 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a024955d-ade1-4d63-9aae-653e6d80cc60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:05.188354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:05.189344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:05.190251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:05.190251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:05.190251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:05.190251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:05.195268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 443/895] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:05.196780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409420e-5e05-4021-8131-5e040d2f8322 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.197107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409420e-5e05-4021-8131-5e040d2f8322 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.197397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409420e-5e05-4021-8131-5e040d2f8322 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.198803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 444/896] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:05.200219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409420e-5e05-4021-8131-5e040d2f8322 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.212586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39481209-48e5-452c-80b3-95029d861426 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.249966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759d198-d662-4444-b312-f832f3aa25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.250622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759d198-d662-4444-b312-f832f3aa25f0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.250906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759d198-d662-4444-b312-f832f3aa25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.251212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759d198-d662-4444-b312-f832f3aa25f0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.251874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759d198-d662-4444-b312-f832f3aa25f0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.253247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759d198-d662-4444-b312-f832f3aa25f0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c759d198-d662-4444-b312-f832f3aa25f0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:05.262640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:05.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:05.264634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 453/897] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:05.265458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d930a-ea5c-4aba-a6e1-0b85262fbcba None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.265458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d930a-ea5c-4aba-a6e1-0b85262fbcba None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.265586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d930a-ea5c-4aba-a6e1-0b85262fbcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.284329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8c9d930a-ea5c-4aba-a6e1-0b85262fbcba None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:05.285579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8c9d930a-ea5c-4aba-a6e1-0b85262fbcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:05.310240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409420e-5e05-4021-8131-5e040d2f8322 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.339410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.344135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b61b59-19a8-46b8-b531-0fcd61713a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.344787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61b59-19a8-46b8-b531-0fcd61713a65 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.345124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61b59-19a8-46b8-b531-0fcd61713a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.352191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61b59-19a8-46b8-b531-0fcd61713a65 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.352191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b61b59-19a8-46b8-b531-0fcd61713a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:05.352191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b61b59-19a8-46b8-b531-0fcd61713a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.352191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b61b59-19a8-46b8-b531-0fcd61713a65 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.352971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 454/898] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:05.355369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b56507-9897-4f0a-8a3c-b1a4f30b9388 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.355369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b56507-9897-4f0a-8a3c-b1a4f30b9388 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.355536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b56507-9897-4f0a-8a3c-b1a4f30b9388 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.356764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 455/899] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:05.357455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b56507-9897-4f0a-8a3c-b1a4f30b9388 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.363166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 445/900] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:05.364706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788fb10-9111-41de-b07a-52d009674664 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.365166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788fb10-9111-41de-b07a-52d009674664 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.365622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788fb10-9111-41de-b07a-52d009674664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.386940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6788fb10-9111-41de-b07a-52d009674664 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:05.388440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6788fb10-9111-41de-b07a-52d009674664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:05.405318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.455032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 446/901] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:05.455945 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788fb10-9111-41de-b07a-52d009674664 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.474263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b56507-9897-4f0a-8a3c-b1a4f30b9388 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.511103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.511800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.512040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.512310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.512999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:05.513306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.514678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.519590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 456/902] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:05.520371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.561206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788fb10-9111-41de-b07a-52d009674664 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.584504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.595909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.596493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.596778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.597054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.604744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.606202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:05.619049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:05.620759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:05.621698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 447/903] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:05.622372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.628143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1623aae-47d4-42ec-b38f-113f4c2d24f6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.635955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.661930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.662567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.662844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.663121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.663770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.665115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:05.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:05.675925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:05.676943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 457/904] 10.222.0.49 () {56 vars in 1132 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:05.677955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.728220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacb1a3-a344-4ae8-af54-530db246f7c1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.763207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.763851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.764138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.764415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.765102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.766478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.781428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b79ce3-c105-4a81-807b-a7c3ff576654 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.814548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.815157 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.815471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.815749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.816416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:05.816721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.818072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:05.822725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 458/905] 10.222.0.49 () {56 vars in 1114 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:05.823593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.846473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 448/906] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:05 2022] POST /identity/v3/services => generated 241 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:05.847266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:05.865906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.901133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:05.943688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6b713-398d-4dda-ba33-bdd76b1a7fbf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.977679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2fcad-3c3f-40cf-aba7-7f73f4e7e73e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:05.984012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:05.984677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:05.984957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:05.985233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:05.986039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:05.986349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:05.987837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.014271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 459/907] 10.222.0.49 () {56 vars in 1149 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:06.014271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.026312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.026983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.027255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.027534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.028212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:06.028521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.030737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.035156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 449/908] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:06.039363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.112739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:06.121771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0e81f-9589-4acf-8855-59b5ab9e0fec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.145686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bced1b-451d-44cb-9599-a79d75e057e8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.147702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:06.158219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.158846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.159129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.159459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.160173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.161535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.183499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.184131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.184419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.184699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.253451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.254950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.303220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 460/909] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:06.303220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.387766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 450/910] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:06.389549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.392014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:06.467500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c7fe5-7751-41a0-9d0f-76c876a6bca1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.504482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.505953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.506385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.506811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.507978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:06.508884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.510660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.513575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20c910-d5c1-4799-b180-5b83c2777a72 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.517636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 461/911] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:06.524710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.541588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:06.555702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.556360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.557068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.557068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.567727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.569235 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:06.585345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:06.586537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:06.587558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:06.587558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:06.587558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:06.587558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:06.587558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 451/912] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:06.593248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.656118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:06.671376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fd68f-4da3-4a9c-9d81-8477053a3216 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.700352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb0093-a3ba-44bf-915e-369d2d3a04ee None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.721537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.722674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.722959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.723529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.724351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] RBAC: Adding query filter params (interface=public, service_id=4430e05251e74ff9b20c326c9d3c8ed5, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:06.724857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.726834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.739285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 462/913] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/endpoints?service_id=4430e05251e74ff9b20c326c9d3c8ed5&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:06.742756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.744704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.745597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.745895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.746157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.746855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:06.747160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.748588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.766206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 452/914] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:06.769387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:06.840773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:06.873596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd4725-a090-486d-a431-2359f426b733 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.903023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a98c2-7077-4ec0-8da5-0ef6107f33e2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:06.918873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:06.921523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.922526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.923324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.923801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.924699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.926693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.935814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 463/915] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:06.940191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:06.941204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:06.941713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:06.942183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:06.951217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:06.953161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:06.966877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:06.967872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:06.968780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:06.975365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 453/916] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:06.977710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:07.097181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd4a1b-ba94-4b52-82f1-b28cac7b41c5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:07.135445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98556e6e-1180-4151-90e8-0d7e19b08f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:07.136063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98556e6e-1180-4151-90e8-0d7e19b08f7e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:07.136345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98556e6e-1180-4151-90e8-0d7e19b08f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:07.136618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98556e6e-1180-4151-90e8-0d7e19b08f7e None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:07.137366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98556e6e-1180-4151-90e8-0d7e19b08f7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:07.137675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98556e6e-1180-4151-90e8-0d7e19b08f7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:07.139101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98556e6e-1180-4151-90e8-0d7e19b08f7e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:07.142273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:07.153438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 464/917] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:07.159520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:07.265330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5393ce-754b-4ac8-ab99-597688743e72 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:07.301441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf305e-de31-4a0b-ab31-949090fa1013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:07.302067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf305e-de31-4a0b-ab31-949090fa1013 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:07.302349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf305e-de31-4a0b-ab31-949090fa1013 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:07.302626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf305e-de31-4a0b-ab31-949090fa1013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:07.303440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf305e-de31-4a0b-ab31-949090fa1013 None admin] RBAC: Adding query filter params (role.id=e14505fd8ab54e46b1b6f3e87b003e87, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=a75c1093580244409a9074cc0d277af1) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:07.303754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf305e-de31-4a0b-ab31-949090fa1013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:07.305278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf305e-de31-4a0b-ab31-949090fa1013 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:07.315273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 454/918] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:07 2022] GET /identity/v3/role_assignments?user.id=a75c1093580244409a9074cc0d277af1&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=e14505fd8ab54e46b1b6f3e87b003e87 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:07.962538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec96d2-4e9a-4821-b194-460d214e74d0 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:07.963592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec96d2-4e9a-4821-b194-460d214e74d0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:07.963672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec96d2-4e9a-4821-b194-460d214e74d0 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:07.965289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 465/919] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:07.970714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1ae73-e3b4-4bea-a9ce-98161f685828 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:07.971107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1ae73-e3b4-4bea-a9ce-98161f685828 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:07.971315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1ae73-e3b4-4bea-a9ce-98161f685828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:07.990387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-71e1ae73-e3b4-4bea-a9ce-98161f685828 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:07.991617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-71e1ae73-e3b4-4bea-a9ce-98161f685828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:08.023881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.063373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 455/920] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:08.085278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec96d2-4e9a-4821-b194-460d214e74d0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.198170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec96d2-4e9a-4821-b194-460d214e74d0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.236556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.237121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.237450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.237733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.238360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.239771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:08.249768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:08.252054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:08.254070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 466/921] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:08.258506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1ae73-e3b4-4bea-a9ce-98161f685828 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.357415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1ae73-e3b4-4bea-a9ce-98161f685828 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.394658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f97e27-c05d-4961-ae56-2bd45a54c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.395318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f97e27-c05d-4961-ae56-2bd45a54c43c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.395610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f97e27-c05d-4961-ae56-2bd45a54c43c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.395885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f97e27-c05d-4961-ae56-2bd45a54c43c None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.396661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f97e27-c05d-4961-ae56-2bd45a54c43c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:08.396969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f97e27-c05d-4961-ae56-2bd45a54c43c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.398362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f97e27-c05d-4961-ae56-2bd45a54c43c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.404311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 456/922] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:08.410026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.478092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.506970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6242c-6a5b-41a6-b053-471bf57c9c11 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.507229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6242c-6a5b-41a6-b053-471bf57c9c11 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.507574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6242c-6a5b-41a6-b053-471bf57c9c11 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.510401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 457/923] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:08.515914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183baa20-0c05-435d-a079-7bfed283e54d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.516554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183baa20-0c05-435d-a079-7bfed283e54d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.517048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183baa20-0c05-435d-a079-7bfed283e54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.531471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3bf51-7323-43b6-acf9-d9e00851e36f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.546319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-183baa20-0c05-435d-a079-7bfed283e54d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:08.548268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-183baa20-0c05-435d-a079-7bfed283e54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:08.571744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.572905 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.573192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.573470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.574155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:08.574459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.576198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.580082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.583155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 467/924] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:08.588705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.643386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 458/925] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:08.660410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183baa20-0c05-435d-a079-7bfed283e54d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.711911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd0227-1ef0-44c8-b71a-9e872fe77b49 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.741428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.755902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.760005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.760005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.760005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.760005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] RBAC: Adding query filter params (interface=public, service_id=ef89b3a90d3c4635b14a69c96f2a1114, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:08.760005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.763108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.770077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 468/926] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/endpoints?service_id=ef89b3a90d3c4635b14a69c96f2a1114&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:08.776635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.801124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183baa20-0c05-435d-a079-7bfed283e54d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.843842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.844847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.845335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.845868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.846749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.848536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.851435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING keystone.server.flask.common [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 10:02:08.886095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41eeb-7a36-4e70-970e-232bcebfedc6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:08.907350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.sql.core [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:02:08.907350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 10:02:08.907350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [parameters: {'id': 'bb3f982d365446a290a1e294001b9d68', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 10:02:08.907350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80467) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb3f982d365446a290a1e294001b9d68.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb3f982d365446a290a1e294001b9d68. Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:02:08.924807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result.read() Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 02 10:02:08.925698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application yield resource Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise self.value Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application session.commit() Aug 02 10:02:08.926679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:02:08.928266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:02:08.929177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result.read() Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:02:08.931158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [parameters: {'id': 'bb3f982d365446a290a1e294001b9d68', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:08.933854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:08.935828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb3f982d365446a290a1e294001b9d68. Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 459/927] 10.222.0.49 () {60 vars in 1140 bytes} [Tue Aug 2 10:02:08 2022] POST /identity/v3/projects => generated 262 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:08.939215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:08.944341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e2aa2-1882-4180-a246-88d29de0448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:08.945769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2aa2-1882-4180-a246-88d29de0448c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:08.946266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2aa2-1882-4180-a246-88d29de0448c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:08.946748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2aa2-1882-4180-a246-88d29de0448c None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:08.949030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2aa2-1882-4180-a246-88d29de0448c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:08.951066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2aa2-1882-4180-a246-88d29de0448c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:08.955491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:08.962239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 469/928] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:09.051905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1deb8c-f16a-4a72-9248-31c7c6de35e0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.064983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1689ed4-3a08-4595-af5b-ec08650d569a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.065507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1689ed4-3a08-4595-af5b-ec08650d569a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.065507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1689ed4-3a08-4595-af5b-ec08650d569a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.067239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 470/929] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:09.073483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53787182-db77-4f91-ab0a-b6be6e989517 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.073769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53787182-db77-4f91-ab0a-b6be6e989517 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.074399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53787182-db77-4f91-ab0a-b6be6e989517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.094374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.095048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.095374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.095659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.098257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-53787182-db77-4f91-ab0a-b6be6e989517 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:09.100025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-53787182-db77-4f91-ab0a-b6be6e989517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:09.105129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.106938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:09.124984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:09.125942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:09.126826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:09.127298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 460/930] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:09.133964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.161774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.188545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 471/931] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:09.211138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53787182-db77-4f91-ab0a-b6be6e989517 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.236244 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.263860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ffbc0-13dd-4a0e-bfe0-3a42c59ca192 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.311058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.311811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.312196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.312749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.313587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:09.314002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.316510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.327631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53787182-db77-4f91-ab0a-b6be6e989517 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.351221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 461/932] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:09.377532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255dc15-c356-49eb-9a2e-9283fde09c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.378280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255dc15-c356-49eb-9a2e-9283fde09c0e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.378582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255dc15-c356-49eb-9a2e-9283fde09c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.378916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255dc15-c356-49eb-9a2e-9283fde09c0e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.379601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255dc15-c356-49eb-9a2e-9283fde09c0e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.381038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255dc15-c356-49eb-9a2e-9283fde09c0e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e255dc15-c356-49eb-9a2e-9283fde09c0e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:09.391497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:09.392492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:09.393741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 472/933] 10.222.0.49 () {56 vars in 1132 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:09.399851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.435472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90496e-964a-4fae-90bc-2fffa9287e7f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.435800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90496e-964a-4fae-90bc-2fffa9287e7f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.436037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90496e-964a-4fae-90bc-2fffa9287e7f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.437767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 473/934] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:09.443257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcae16f-a507-4356-b72c-a2ad5d34b45a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.443548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcae16f-a507-4356-b72c-a2ad5d34b45a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.443816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcae16f-a507-4356-b72c-a2ad5d34b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.464477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-edcae16f-a507-4356-b72c-a2ad5d34b45a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:09.465762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-edcae16f-a507-4356-b72c-a2ad5d34b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:09.533225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.540956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64696cba-6c52-479c-bec2-07d73b0d9a43 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.556396 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 474/935] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:09.581561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcae16f-a507-4356-b72c-a2ad5d34b45a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.583111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.583902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.584193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.584473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.585160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:09.586061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.589340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.595343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 462/936] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:09.601451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.606298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.715401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcae16f-a507-4356-b72c-a2ad5d34b45a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.731969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb1fd-b5c9-4cdc-937b-02f91bcac6b4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.766447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.767793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.768260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.768714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.771258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.771960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.772315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.772761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.773413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:09.773789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.775264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.783837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:09.786535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 463/937] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:09.789772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:09.789896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:09.790855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:09.792305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:09.795048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 475/938] 10.222.0.49 () {56 vars in 1120 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:09.800237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:09.839835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.878005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.919334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5e39-78e4-4e16-9614-1190943599f1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.924940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b60f3-a5cf-4014-8d19-8b1088b53b2c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:09.962089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.962422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:09.962777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.963611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:09.963719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.963921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:09.964021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.964201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:09.964656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.964926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:09.965238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:09.966024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.966592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:09.976344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 476/939] 10.222.0.49 () {56 vars in 1115 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:09.983043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.083333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 464/940] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:10.093541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:10.099162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27a567-2969-4e94-ba12-6aacb9b75952 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.149737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.151614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.152107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.152586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.167212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.169180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:10.188664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:10.189623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:10.190554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:10.190554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:10.190554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:10.190554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:10.193063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 477/941] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:10.199279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.310828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e991-a9df-474c-be30-8e1afcc37d0c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.334667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:10.352088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.352088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.352088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.352088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.352512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:10.352512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.352512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.364510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 465/942] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:10.370613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.475596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a571b-7396-4935-89f5-2b93f09e6efe None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.516931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.517592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.517944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.518262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.529156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.530917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:10.542254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:10.543511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:10.544854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 478/943] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:02:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:10.552626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.676848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754e5e-c793-434c-9841-ccdea3aef88e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.714309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.714959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.715286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.715566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.716372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:10.716682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.718124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.738375 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 466/944] 10.222.0.49 () {56 vars in 1149 bytes} [Tue Aug 2 10:02:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:10.744034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.754767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:10.851978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca199e8-9f89-4172-81e0-85a600ecefce None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:10.891815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:10.892470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:10.892752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:10.893032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:10.904997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:10.906455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:10.921057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:10.921978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:10.922876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:10.922876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:10.922876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:10.922876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:10.922876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 479/945] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:10.929509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:10.979249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.048341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cedb8-acef-4d45-bc32-3e62096a10db None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.088187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70290d92-ce2b-467b-a3a3-478e2148af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.088844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70290d92-ce2b-467b-a3a3-478e2148af09 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.089083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70290d92-ce2b-467b-a3a3-478e2148af09 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.089349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70290d92-ce2b-467b-a3a3-478e2148af09 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.090041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70290d92-ce2b-467b-a3a3-478e2148af09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:11.090342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70290d92-ce2b-467b-a3a3-478e2148af09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.091756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70290d92-ce2b-467b-a3a3-478e2148af09 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.104107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 467/946] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:11.109958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.160037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb206-c80f-4089-9359-c4174c5a66d3 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.160374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb206-c80f-4089-9359-c4174c5a66d3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.160597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb206-c80f-4089-9359-c4174c5a66d3 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.162122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 468/947] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:11.166964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd75bd-b1e6-4b44-8e60-a5b9b48989f8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.167277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd75bd-b1e6-4b44-8e60-a5b9b48989f8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.167562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd75bd-b1e6-4b44-8e60-a5b9b48989f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.175688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.191754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-46fd75bd-b1e6-4b44-8e60-a5b9b48989f8 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:11.191754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-46fd75bd-b1e6-4b44-8e60-a5b9b48989f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:11.223871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac014e7-6772-4270-ac85-f7eccf24183d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.261990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.263015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.263543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.264055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.267430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 469/948] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:11.272018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.273717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.282239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd75bd-b1e6-4b44-8e60-a5b9b48989f8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:11.287575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:11.288496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:11.289425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:11.292134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 480/949] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:11.298980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.318763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.408641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd75bd-b1e6-4b44-8e60-a5b9b48989f8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.423146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c4347-c03f-4a83-adf1-ad9f214e536f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.441684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.463635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.463635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.463635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.463982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.464073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.464160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.464234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.464310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.464310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.464310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:11.464517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.464589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.464589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:11.474560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:11.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:11.476804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:11.477606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 481/950] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:11.487462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 470/951] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:11.489691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.490197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.605608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.613221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73131c-bc36-4475-9bc0-15ed449d1c21 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.637643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66f4fe-dbb4-47ca-8fef-4a3ebd584044 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.650377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10bb4c-288a-4025-8912-e984018c8f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.651608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10bb4c-288a-4025-8912-e984018c8f4c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.651899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10bb4c-288a-4025-8912-e984018c8f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.652224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10bb4c-288a-4025-8912-e984018c8f4c None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.652900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10bb4c-288a-4025-8912-e984018c8f4c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:11.653208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10bb4c-288a-4025-8912-e984018c8f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.654593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10bb4c-288a-4025-8912-e984018c8f4c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.659664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 471/952] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:11.660904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c231b-ffe7-4b7d-b1fd-7df14731b4c0 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.661194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c231b-ffe7-4b7d-b1fd-7df14731b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.661477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c231b-ffe7-4b7d-b1fd-7df14731b4c0 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.662865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 472/953] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:11.667591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c231b-ffe7-4b7d-b1fd-7df14731b4c0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.679716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57a777-bf80-439e-a6e9-5b6c9512ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.680399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57a777-bf80-439e-a6e9-5b6c9512ab69 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.680686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57a777-bf80-439e-a6e9-5b6c9512ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.680968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57a777-bf80-439e-a6e9-5b6c9512ab69 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/a75c1093580244409a9074cc0d277af1/roles/e14505fd8ab54e46b1b6f3e87b003e87` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.683178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57a777-bf80-439e-a6e9-5b6c9512ab69 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=a75c1093580244409a9074cc0d277af1, role_id=e14505fd8ab54e46b1b6f3e87b003e87)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.684641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57a777-bf80-439e-a6e9-5b6c9512ab69 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.772942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c231b-ffe7-4b7d-b1fd-7df14731b4c0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.795550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 482/954] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:11 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/a75c1093580244409a9074cc0d277af1/roles/e14505fd8ab54e46b1b6f3e87b003e87 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:11.797137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d1b2a-22fb-4da4-94d1-07a48e632e57 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.797439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d1b2a-22fb-4da4-94d1-07a48e632e57 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.797727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d1b2a-22fb-4da4-94d1-07a48e632e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.811157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:11.811956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:11.812241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:11.812599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:11.813345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:11.813660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:11.815375 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:11.827333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 473/955] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:11.832143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:11.843615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fb4d1b2a-22fb-4da4-94d1-07a48e632e57 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:11.845096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fb4d1b2a-22fb-4da4-94d1-07a48e632e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:11.859266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.915570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:11.954047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65035f-12b5-434b-aa62-cbcbac4c0fd4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:11.963608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 483/956] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:11.991527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d1b2a-22fb-4da4-94d1-07a48e632e57 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:12.000046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb40c0-ad27-447f-99a5-a41f35fb5eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:12.000680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb40c0-ad27-447f-99a5-a41f35fb5eec None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.001007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb40c0-ad27-447f-99a5-a41f35fb5eec None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.001339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb40c0-ad27-447f-99a5-a41f35fb5eec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.002935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb40c0-ad27-447f-99a5-a41f35fb5eec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:12.004428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb40c0-ad27-447f-99a5-a41f35fb5eec None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:12.094197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 474/957] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:12.116351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d1b2a-22fb-4da4-94d1-07a48e632e57 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:12.168467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:12.169613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.170103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.170647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.171592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:12.173432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:12.176644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING keystone.server.flask.common [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 10:02:12.235394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.sql.core [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:02:12.235394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 10:02:12.235394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [parameters: {'user_id': '69dfd9a5351e4e5ea49c5d4d07fbc96f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 10:02:12.235394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80466) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:02:12.241142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7182-be7f-47a1-b8e5-839188791950 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.241438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7182-be7f-47a1-b8e5-839188791950 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.241724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7182-be7f-47a1-b8e5-839188791950 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.243433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 475/958] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() Aug 02 10:02:12.244975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:02:12.246167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application yield resource Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise self.value Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application session.commit() Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:02:12.247162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:02:12.248084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:02:12.249740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:02:12.250591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [parameters: {'user_id': '69dfd9a5351e4e5ea49c5d4d07fbc96f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:12.251457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 02 10:02:12.252564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 02 10:02:12.253722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 02 10:02:12.253722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 02 10:02:12.253722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:02:12.253722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:02:12.253722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 10:02:12.253722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:12.254238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9ae2-b30d-48aa-b65d-b73cdc40fef6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.254238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9ae2-b30d-48aa-b65d-b73cdc40fef6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.254238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9ae2-b30d-48aa-b65d-b73cdc40fef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.256042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 484/959] 10.222.0.49 () {60 vars in 1131 bytes} [Tue Aug 2 10:02:11 2022] POST /identity/v3/users => generated 160 bytes in 265 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:12.262397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:12.273127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-053e9ae2-b30d-48aa-b65d-b73cdc40fef6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:12.274438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-053e9ae2-b30d-48aa-b65d-b73cdc40fef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:12.296168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:12.358921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 476/960] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:12.381272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e9ae2-b30d-48aa-b65d-b73cdc40fef6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:12.392415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:12.397054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed091fe-3b51-47fe-8244-d0a595bd7ee1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:12.442570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:12.443539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.443837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.444120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.454018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:12.455928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:12.469387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:12.470590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:02:12.471573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:12.473758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 485/961] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:12.483227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:12.500592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e9ae2-b30d-48aa-b65d-b73cdc40fef6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:12.538676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:12.539667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.540052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.540461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.541927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:12.543434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:12.552898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:12.554117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:12.555219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:12.562035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 477/962] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:12.563457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:12.593405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:12.606548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbdb84-af80-4644-b68d-8867e19c2c62 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:12.647438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:12.647991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.648279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.648563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.649956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:12.650275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:12.652553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:12.654976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:12.687985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ee272-c7e3-48da-9b84-01854ae6bc33 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:12.703760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 486/963] 10.222.0.49 () {56 vars in 1101 bytes} [Tue Aug 2 10:02:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:12.728112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e57d7-7682-4c5d-be80-7b5ddbbc722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:12.729495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e57d7-7682-4c5d-be80-7b5ddbbc722d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.729999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e57d7-7682-4c5d-be80-7b5ddbbc722d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.730531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e57d7-7682-4c5d-be80-7b5ddbbc722d None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.731489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e57d7-7682-4c5d-be80-7b5ddbbc722d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:12.732030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e57d7-7682-4c5d-be80-7b5ddbbc722d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:12.733765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e57d7-7682-4c5d-be80-7b5ddbbc722d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:12.745396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 478/964] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:12.747460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:12.860007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979b3d7-6349-4d2b-bed3-354f8e13ea2f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:12.897193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf288d3f-6bdf-433f-8ded-82c9f57153eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:12.897869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf288d3f-6bdf-433f-8ded-82c9f57153eb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:12.898102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf288d3f-6bdf-433f-8ded-82c9f57153eb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:12.898385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf288d3f-6bdf-433f-8ded-82c9f57153eb None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:12.899070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf288d3f-6bdf-433f-8ded-82c9f57153eb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:12.899426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf288d3f-6bdf-433f-8ded-82c9f57153eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:12.900828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf288d3f-6bdf-433f-8ded-82c9f57153eb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:12.906052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 487/965] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:13.924518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ce09-9462-49ee-a17a-b5b629351b46 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:13.925046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ce09-9462-49ee-a17a-b5b629351b46 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:13.925046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ce09-9462-49ee-a17a-b5b629351b46 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:13.926528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 479/966] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:13.931586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ba4f-91a8-45df-9681-08a4049383c6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:13.932580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ba4f-91a8-45df-9681-08a4049383c6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:13.932771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ba4f-91a8-45df-9681-08a4049383c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:13.955537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0e57ba4f-91a8-45df-9681-08a4049383c6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:13.957150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e57ba4f-91a8-45df-9681-08a4049383c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:13.959457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.028466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 488/967] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:14.051299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ce09-9462-49ee-a17a-b5b629351b46 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.155607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ce09-9462-49ee-a17a-b5b629351b46 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.188927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba6fa1-d4f9-4e81-ac61-19ac00d9e20a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.189271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba6fa1-d4f9-4e81-ac61-19ac00d9e20a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.189444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba6fa1-d4f9-4e81-ac61-19ac00d9e20a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.190894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 489/968] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:14.191016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.191506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.191795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.192077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.195111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.195497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a294-e652-4691-8402-293d79d207fa None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.195784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a294-e652-4691-8402-293d79d207fa None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.196065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a294-e652-4691-8402-293d79d207fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.205145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:14.210452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:14.219788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:14.220830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 480/969] 10.222.0.49 () {56 vars in 1120 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:14.221564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c225a294-e652-4691-8402-293d79d207fa None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:14.221564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c225a294-e652-4691-8402-293d79d207fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:14.221709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.292854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 490/970] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:14.315650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a294-e652-4691-8402-293d79d207fa None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.329279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad95553-a504-483d-b356-9eaf8b2460f1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.368057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.368979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.369269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.370159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.371158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:14.371509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.373340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.382293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 481/971] 10.222.0.49 () {56 vars in 1115 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:14.388087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.390589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.424094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a294-e652-4691-8402-293d79d207fa None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.454996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.460139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.460795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.461078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.461356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.462042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.463451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:14.472807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:14.473785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:14.474623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 491/972] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:14.480351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.504217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6b232-bcf5-4c5b-8525-7226d37e7d0a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.543377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.543995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.544481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.544772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.557137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.558678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:14.575050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:14.576230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:14.577290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:14.577290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:14.577290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:14.577290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:14.581233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 482/973] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:14.582532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.599784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8ab5d-3f70-4269-bbd8-d64a247a853e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.630679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.639267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.639267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.644857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.644857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.644857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:14.644857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.644857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.654170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 492/974] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:14.655332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.709545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a95d1-9bf0-467d-b3ec-9a23d77b44c9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.720118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.745540 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.746717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.746999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.747304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.748072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:14.748348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.749751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.772445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 483/975] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:14.772631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e79e-0dda-462a-9048-5504dd34ff2b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.773522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.809663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.810294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.810583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.811058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.812701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:14.813038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.814446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.820921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 493/976] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:14.826950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.891519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:14.909172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4588ec-97fe-4299-b0e0-8c54083f6049 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.934167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8378cad-95d7-4da7-82a7-8a26e7a13d14 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:14.949181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858389-0d0c-4566-a8c7-434884108e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.949847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858389-0d0c-4566-a8c7-434884108e93 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.950136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858389-0d0c-4566-a8c7-434884108e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.950405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858389-0d0c-4566-a8c7-434884108e93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.958789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e858389-0d0c-4566-a8c7-434884108e93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.960339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e858389-0d0c-4566-a8c7-434884108e93 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.968785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:14.969410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.969693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.969971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.970729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] RBAC: Adding query filter params (interface=admin, service_id=ef89b3a90d3c4635b14a69c96f2a1114, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:14.971044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-7e858389-0d0c-4566-a8c7-434884108e93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:14.971977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:14.973119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:02:14.974099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:14.974690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:14.974769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 484/977] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:14.974769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1961-64c5-497b-809b-af979aff18cb None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:14.974892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1961-64c5-497b-809b-af979aff18cb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:14.975481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1961-64c5-497b-809b-af979aff18cb None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:14.976509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 485/978] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:14.979146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba1961-64c5-497b-809b-af979aff18cb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:14.984955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 494/979] 10.222.0.49 () {56 vars in 1249 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/endpoints?service_id=ef89b3a90d3c4635b14a69c96f2a1114&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:14.985783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.015547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.087176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba1961-64c5-497b-809b-af979aff18cb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.089428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75de13-71f0-42d3-be79-7a6e4beef292 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.111953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.121617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.122239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.126810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.126810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.127028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb135b-da81-4718-9219-00bdc14aaf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.127123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:15.127251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb135b-da81-4718-9219-00bdc14aaf81 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.127327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.127417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb135b-da81-4718-9219-00bdc14aaf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.127417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb135b-da81-4718-9219-00bdc14aaf81 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.127417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb135b-da81-4718-9219-00bdc14aaf81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.127596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.127674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb135b-da81-4718-9219-00bdc14aaf81 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.132311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 495/980] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:15.133550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0421e-bcfc-4147-81e1-d777aacd9c4e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.133852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0421e-bcfc-4147-81e1-d777aacd9c4e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.134129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0421e-bcfc-4147-81e1-d777aacd9c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.145122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 486/981] 10.222.0.49 () {56 vars in 1149 bytes} [Tue Aug 2 10:02:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:15.151333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.156338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b0c0421e-bcfc-4147-81e1-d777aacd9c4e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:15.156960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b0c0421e-bcfc-4147-81e1-d777aacd9c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:15.236526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 496/982] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:15.258672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.259028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0421e-bcfc-4147-81e1-d777aacd9c4e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.272640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05cf35-951b-4250-9b19-9b41e15da886 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.314872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.316922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.316922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.316922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.377714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0421e-bcfc-4147-81e1-d777aacd9c4e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.394648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.396144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.397956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.415367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.415997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.416285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.416599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.417301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.418680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.485278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.495395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 497/983] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:15 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:15.499387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 487/984] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:15.505547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.630864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd861c9-3ae0-4175-b47d-85092b6ed018 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.672859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.673519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.673802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.674078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.686097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.687662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:15.705808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:15.707146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:15.708229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:15.708229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:15.708229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:15.708229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:15.708229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 498/985] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:15.714398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.824125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306d43-287c-4153-916a-b55ce6f9eec6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:15.858649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:15.859306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:15.859589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:15.859864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:15.860553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:15.860855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:15.862219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:15.875586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 488/986] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:15.880821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:15.910159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:15.985297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d564f-05be-4637-b622-6e2e2b0c0d22 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.023624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.024315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.024603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.024876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.031687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.033145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:16.044814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:16.045751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:16.051921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 499/987] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:16.052435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.102354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:16.155534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d309-dbb6-4255-9182-a8d611b3553b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.190571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a9344-1159-4dd4-b1cd-12b0622e062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.191202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a9344-1159-4dd4-b1cd-12b0622e062f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.191503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a9344-1159-4dd4-b1cd-12b0622e062f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.191774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a9344-1159-4dd4-b1cd-12b0622e062f None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.192518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a9344-1159-4dd4-b1cd-12b0622e062f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:16.192827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a9344-1159-4dd4-b1cd-12b0622e062f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.194252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a9344-1159-4dd4-b1cd-12b0622e062f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.206619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 489/988] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:16.212625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:16.308488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:16.323373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccb991-a574-4017-8ea1-5bed5e06ef1a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:16.357906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cd0b5-12ef-49bb-a181-149ef8bc3e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:16.358462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd0b5-12ef-49bb-a181-149ef8bc3e66 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:16.358745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd0b5-12ef-49bb-a181-149ef8bc3e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:16.359025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd0b5-12ef-49bb-a181-149ef8bc3e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:16.359844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd0b5-12ef-49bb-a181-149ef8bc3e66 None admin] RBAC: Adding query filter params (role.id=e14505fd8ab54e46b1b6f3e87b003e87, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=a75c1093580244409a9074cc0d277af1) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:16.360158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd0b5-12ef-49bb-a181-149ef8bc3e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:16.361587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd0b5-12ef-49bb-a181-149ef8bc3e66 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:16.373591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 500/989] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:16 2022] GET /identity/v3/role_assignments?user.id=a75c1093580244409a9074cc0d277af1&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=e14505fd8ab54e46b1b6f3e87b003e87 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:17.177132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe172f-45fd-4f5e-81ac-afb5c9d2b586 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.177786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe172f-45fd-4f5e-81ac-afb5c9d2b586 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.177786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe172f-45fd-4f5e-81ac-afb5c9d2b586 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.179268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 490/990] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:17.184987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bebe5-a8cc-4c8b-8c2d-029f681dc131 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.185419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bebe5-a8cc-4c8b-8c2d-029f681dc131 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.185601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bebe5-a8cc-4c8b-8c2d-029f681dc131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.204059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-278bebe5-a8cc-4c8b-8c2d-029f681dc131 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:17.205267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-278bebe5-a8cc-4c8b-8c2d-029f681dc131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:17.275232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 501/991] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:17.297492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe172f-45fd-4f5e-81ac-afb5c9d2b586 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.401702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe172f-45fd-4f5e-81ac-afb5c9d2b586 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.411388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.437503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4446e8-0a6a-49bc-af6c-1a225cf89c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.438073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4446e8-0a6a-49bc-af6c-1a225cf89c94 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.438451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4446e8-0a6a-49bc-af6c-1a225cf89c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.438732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4446e8-0a6a-49bc-af6c-1a225cf89c94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.439393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4446e8-0a6a-49bc-af6c-1a225cf89c94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.440882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4446e8-0a6a-49bc-af6c-1a225cf89c94 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6c4446e8-0a6a-49bc-af6c-1a225cf89c94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:17.450943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:17.452192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:02:17.453146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:17.456211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 491/992] 10.222.0.49 () {56 vars in 1126 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:17.462046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bebe5-a8cc-4c8b-8c2d-029f681dc131 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.556948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.569056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e108bfa-255c-46b0-a473-2b73bfc9e76e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.575977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e108bfa-255c-46b0-a473-2b73bfc9e76e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.575977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e108bfa-255c-46b0-a473-2b73bfc9e76e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.575977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 492/993] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:17.576365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525de0a-4b52-4b7c-9bdd-830e8e7cc77b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.576654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525de0a-4b52-4b7c-9bdd-830e8e7cc77b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.576935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525de0a-4b52-4b7c-9bdd-830e8e7cc77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.578433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bebe5-a8cc-4c8b-8c2d-029f681dc131 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.596360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3525de0a-4b52-4b7c-9bdd-830e8e7cc77b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:17.597568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3525de0a-4b52-4b7c-9bdd-830e8e7cc77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:17.615715 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.616784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.617294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.617865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.618693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:17.619163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.620652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.626601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 502/994] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:17.631994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.675047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 493/995] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:17.698686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525de0a-4b52-4b7c-9bdd-830e8e7cc77b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.744412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcb3f8-e54f-4ec9-a079-be02f03ab121 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.782577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.783217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.783512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.783785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.784461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:17.784776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.786143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.798862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 503/996] 10.222.0.49 () {56 vars in 1122 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:17.799749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.805768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525de0a-4b52-4b7c-9bdd-830e8e7cc77b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.808044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.811353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:17.844922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.845609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.845895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.846178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.846809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.848307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:17.857804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:17.858952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:17.859833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:17.864227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 494/997] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:17.865066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:17.909537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926bcb-5f71-48fd-b020-3106c862e3ab None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:17.945970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:17.946705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:17.946970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:17.947279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:17.948544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:17.949969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:17.972248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ff0de-ac21-4a51-b3ec-07b7b69e8d01 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.006000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.006702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.006964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.007272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.007953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.008265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.009645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.016493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 495/998] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.023232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 504/999] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:18.024622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.072917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.153706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14632ad0-1855-4b70-9a30-d6afdb3356c2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.201642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.201642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.202331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.202331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.203147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.203316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.204620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.209814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 496/1000] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.215949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.289733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.324972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50a0f-b7c0-4c0c-9fe3-d66b2bc8ad35 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.359981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74621c23-146a-463a-8b66-9d14c05b9ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.360611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74621c23-146a-463a-8b66-9d14c05b9ecc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.360893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74621c23-146a-463a-8b66-9d14c05b9ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.361170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74621c23-146a-463a-8b66-9d14c05b9ecc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.361951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74621c23-146a-463a-8b66-9d14c05b9ecc None admin] RBAC: Adding query filter params (interface=public, service_id=75755279bc024f12b56546ae6e9b453b, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.362264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74621c23-146a-463a-8b66-9d14c05b9ecc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.363777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74621c23-146a-463a-8b66-9d14c05b9ecc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.374461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 505/1001] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/endpoints?service_id=75755279bc024f12b56546ae6e9b453b&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.375721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.424846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256379da-ec23-4cce-9b41-85b648c8afbb None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.425143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256379da-ec23-4cce-9b41-85b648c8afbb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.425419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256379da-ec23-4cce-9b41-85b648c8afbb None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.427029 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 506/1002] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:18.431733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac9164-6e5a-4b11-88a8-790391d0d498 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.432022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac9164-6e5a-4b11-88a8-790391d0d498 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.432350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac9164-6e5a-4b11-88a8-790391d0d498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.452194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-edac9164-6e5a-4b11-88a8-790391d0d498 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:18.453325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-edac9164-6e5a-4b11-88a8-790391d0d498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:18.472093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.485566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ebae-5c23-49fb-9ce9-551123fd5c16 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.530137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.530137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.530137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.530551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.531098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.535223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.535361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 507/1003] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:18 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:18.543224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 497/1004] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:18.553769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac9164-6e5a-4b11-88a8-790391d0d498 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.623149 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:18.665863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac9164-6e5a-4b11-88a8-790391d0d498 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.704991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.705683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.705935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.706206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] PATH_INFO: `/v3/services/image` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.706837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.708225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:18.717907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:18.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:18.719804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 508/1005] 10.222.0.49 () {56 vars in 1105 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:18.725100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:18.862932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ba73-7712-4aa2-9f43-23b62e3f8a34 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:18.898522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595984f4-5f81-4f90-9616-39d8b2ad0531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:18.899150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595984f4-5f81-4f90-9616-39d8b2ad0531 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:18.899465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595984f4-5f81-4f90-9616-39d8b2ad0531 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:18.899745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595984f4-5f81-4f90-9616-39d8b2ad0531 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:18.900428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595984f4-5f81-4f90-9616-39d8b2ad0531 None admin] RBAC: Adding query filter params (name=image) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:18.900739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595984f4-5f81-4f90-9616-39d8b2ad0531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:18.902648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595984f4-5f81-4f90-9616-39d8b2ad0531 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:18.907912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 498/1006] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:18.913331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.019777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ffa8-2312-4e28-b6fe-1bd68de8ec48 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.045483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.054566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32435823-f37a-49eb-abf9-508d3315381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.059501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32435823-f37a-49eb-abf9-508d3315381f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.059501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32435823-f37a-49eb-abf9-508d3315381f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.059501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32435823-f37a-49eb-abf9-508d3315381f None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.059501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32435823-f37a-49eb-abf9-508d3315381f None admin] RBAC: Adding query filter params (type=image) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:19.059501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32435823-f37a-49eb-abf9-508d3315381f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.059501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32435823-f37a-49eb-abf9-508d3315381f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.071233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 509/1007] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:20.092804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20fc89-4b26-4384-a701-e93466380058 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.093336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20fc89-4b26-4384-a701-e93466380058 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.093417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20fc89-4b26-4384-a701-e93466380058 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.094905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 499/1008] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:20.100121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d7dbb-c388-4808-8c91-eb92827cb6dc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.100507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d7dbb-c388-4808-8c91-eb92827cb6dc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.100682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d7dbb-c388-4808-8c91-eb92827cb6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.120777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d18d7dbb-c388-4808-8c91-eb92827cb6dc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:20.122255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d18d7dbb-c388-4808-8c91-eb92827cb6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:20.194598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 510/1009] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:20.217735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20fc89-4b26-4384-a701-e93466380058 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.238365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.324490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20fc89-4b26-4384-a701-e93466380058 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.358435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2190e-f2d9-4d12-8685-2666ddcc2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.358999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2190e-f2d9-4d12-8685-2666ddcc2414 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.359312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2190e-f2d9-4d12-8685-2666ddcc2414 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.359595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2190e-f2d9-4d12-8685-2666ddcc2414 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.360746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2190e-f2d9-4d12-8685-2666ddcc2414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.362186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2190e-f2d9-4d12-8685-2666ddcc2414 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.363973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 500/1010] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:20.369741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d7dbb-c388-4808-8c91-eb92827cb6dc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.470305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d7dbb-c388-4808-8c91-eb92827cb6dc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.490832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.511215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.511215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.511215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.511215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.511669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.511992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.567586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b030f1d-5e21-4fea-82ff-a04acbead89a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.567858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b030f1d-5e21-4fea-82ff-a04acbead89a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.568140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b030f1d-5e21-4fea-82ff-a04acbead89a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.569644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 501/1011] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:20.574493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160caa61-1b0d-4901-8b73-7edfa2a26bb6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.574779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160caa61-1b0d-4901-8b73-7edfa2a26bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.575059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160caa61-1b0d-4901-8b73-7edfa2a26bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.593064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-160caa61-1b0d-4901-8b73-7edfa2a26bb6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:20.594241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-160caa61-1b0d-4901-8b73-7edfa2a26bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:20.670011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 511/1012] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:20 2022] POST /identity/v3/projects => generated 311 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:20.670204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 502/1013] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:20.689132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.800171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e10532-cdae-4f8c-b5e6-67d6fe403276 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:20.841452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:20.842214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:20.842467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:20.842739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:20.843392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:20.844748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:20.854191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:20.855910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:20.856763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 512/1014] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:20.862106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160caa61-1b0d-4901-8b73-7edfa2a26bb6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:20.882337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:20.971037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160caa61-1b0d-4901-8b73-7edfa2a26bb6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.007534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe1462-c620-4336-bfbb-0f58024b3322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.008414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe1462-c620-4336-bfbb-0f58024b3322 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.008849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe1462-c620-4336-bfbb-0f58024b3322 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.009303 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe1462-c620-4336-bfbb-0f58024b3322 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.010122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe1462-c620-4336-bfbb-0f58024b3322 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:21.010610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe1462-c620-4336-bfbb-0f58024b3322 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.012148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe1462-c620-4336-bfbb-0f58024b3322 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.017226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 503/1015] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:21.022979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.061384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.083570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cf8ca-e80c-4040-9a39-e0005967dd07 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.083570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cf8ca-e80c-4040-9a39-e0005967dd07 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.083570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cf8ca-e80c-4040-9a39-e0005967dd07 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.083570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 504/1016] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:21.090678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2a45c-2bde-4a44-82dc-2be98a44cacf None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.090979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2a45c-2bde-4a44-82dc-2be98a44cacf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.091295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2a45c-2bde-4a44-82dc-2be98a44cacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.110317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d6b2a45c-2bde-4a44-82dc-2be98a44cacf None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:21.111526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d6b2a45c-2bde-4a44-82dc-2be98a44cacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:21.143762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7ab9-871c-43f8-aa57-d3ee4d67b42e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.178443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.179061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.179407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.179689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.180368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:21.180673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.182049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.191243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 513/1017] 10.222.0.49 () {56 vars in 1112 bytes} [Tue Aug 2 10:02:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:21.191422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 505/1018] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:21.192739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.207202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2a45c-2bde-4a44-82dc-2be98a44cacf None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:21.219301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.298303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:21.302464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf737d-e47a-498a-ab57-450a2074b218 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.321175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2a45c-2bde-4a44-82dc-2be98a44cacf None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:21.361972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505f413-b02d-494e-bf7a-3c6dff383fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.362973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505f413-b02d-494e-bf7a-3c6dff383fb6 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.363472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505f413-b02d-494e-bf7a-3c6dff383fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.363935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505f413-b02d-494e-bf7a-3c6dff383fb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.365150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505f413-b02d-494e-bf7a-3c6dff383fb6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.367478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505f413-b02d-494e-bf7a-3c6dff383fb6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.382495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3b75a-a72b-4bc6-b568-00f4e4961aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:21.383451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3b75a-a72b-4bc6-b568-00f4e4961aed None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:21.383744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3b75a-a72b-4bc6-b568-00f4e4961aed None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:21.384042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3b75a-a72b-4bc6-b568-00f4e4961aed None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:21.385135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3b75a-a72b-4bc6-b568-00f4e4961aed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:21.386617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3b75a-a72b-4bc6-b568-00f4e4961aed None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:21.450059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 514/1019] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:21.471676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 506/1020] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:21 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:22.769728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892fd4b-dd01-4142-bf88-f2c475f40e45 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.770245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892fd4b-dd01-4142-bf88-f2c475f40e45 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.770325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892fd4b-dd01-4142-bf88-f2c475f40e45 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.771733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 515/1021] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:22.777044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4544e-f9d9-457e-975d-dbc7ee1d232e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.777453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4544e-f9d9-457e-975d-dbc7ee1d232e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.777623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4544e-f9d9-457e-975d-dbc7ee1d232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.796218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-abb4544e-f9d9-457e-975d-dbc7ee1d232e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:22.797319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-abb4544e-f9d9-457e-975d-dbc7ee1d232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.859424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:22.871039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 507/1022] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:22.895233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892fd4b-dd01-4142-bf88-f2c475f40e45 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.999506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892fd4b-dd01-4142-bf88-f2c475f40e45 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.033337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185bee4-8440-458d-84c2-0912c7604bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.033983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185bee4-8440-458d-84c2-0912c7604bfe None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.034262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185bee4-8440-458d-84c2-0912c7604bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.034534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185bee4-8440-458d-84c2-0912c7604bfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.035754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185bee4-8440-458d-84c2-0912c7604bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.037247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185bee4-8440-458d-84c2-0912c7604bfe None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.039037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 516/1023] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:23.045463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4544e-f9d9-457e-975d-dbc7ee1d232e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.148582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4544e-f9d9-457e-975d-dbc7ee1d232e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.184097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e0996-61fd-4cc0-a240-ed8642815a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.184774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0996-61fd-4cc0-a240-ed8642815a26 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.184988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0996-61fd-4cc0-a240-ed8642815a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.185262 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0996-61fd-4cc0-a240-ed8642815a26 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.185952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e0996-61fd-4cc0-a240-ed8642815a26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.187515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e0996-61fd-4cc0-a240-ed8642815a26 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.278931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 508/1024] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:23.541795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2880-d4a9-47e9-918d-16cf500e959c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.542129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2880-d4a9-47e9-918d-16cf500e959c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.542418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2880-d4a9-47e9-918d-16cf500e959c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.543875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 517/1025] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:23.549081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62aafde-1653-443d-9a96-fdb0120b5495 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.549381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62aafde-1653-443d-9a96-fdb0120b5495 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.549661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62aafde-1653-443d-9a96-fdb0120b5495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.568184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b62aafde-1653-443d-9a96-fdb0120b5495 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:23.569386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b62aafde-1653-443d-9a96-fdb0120b5495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.608851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f367a51-76ff-4154-b2bb-8287842338e2 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.609176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f367a51-76ff-4154-b2bb-8287842338e2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.609447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f367a51-76ff-4154-b2bb-8287842338e2 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.611106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 518/1026] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:23.616387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e18b-5752-464e-bd49-263ec112f796 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.616678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e18b-5752-464e-bd49-263ec112f796 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.616963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e18b-5752-464e-bd49-263ec112f796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.626823 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.635691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6588e18b-5752-464e-bd49-263ec112f796 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:23.636897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6588e18b-5752-464e-bd49-263ec112f796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.647864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 509/1027] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:23.665093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.670658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62aafde-1653-443d-9a96-fdb0120b5495 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.755255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 519/1028] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:23.779674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e18b-5752-464e-bd49-263ec112f796 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.787992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62aafde-1653-443d-9a96-fdb0120b5495 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.827626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.828283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.828597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.828898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] PATH_INFO: `/v3/services/image` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.829591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.831002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:23.840082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:23.842241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:23.843098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 510/1029] 10.222.0.49 () {56 vars in 1105 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:23.847806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.897132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e18b-5752-464e-bd49-263ec112f796 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.898418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.932360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.933656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.933943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.934218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.944849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:23.949844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:23.950959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:23.951995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 520/1030] 10.222.0.49 () {56 vars in 1120 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:23.956844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.965402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faa5c0-af0d-4635-9484-3eafa6d2eeeb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.974194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.004151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.004782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.005065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.005341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.006772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.007085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.008486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.018342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 511/1031] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:24.019532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.071828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3faf9-0bc0-4f7f-bd4c-14cc24daf6cf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.109255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.109877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.110166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.110677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.111429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.111746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.113130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.122741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 521/1032] 10.222.0.49 () {56 vars in 1115 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:24.128326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.129638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4920c2-a062-4eb0-89c9-f3316c6eccd7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.141724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.166094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.166715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.167600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.167889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.168566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] RBAC: Adding query filter params (type=image) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.168878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.170340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.180728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 512/1033] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:24.181708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.213525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.239751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e3416-a29e-4486-b05c-16a0e955ddaf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.275530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.276172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.276682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.276791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.288050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.289140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.296703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2d4d1-bb7d-4cdc-83d3-681ae0e202ae None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:24.304646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:24.305794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:24.306691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:24.306691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:24.306691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:24.306691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:24.306691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 522/1034] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:24.312026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.416878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.417979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.418321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.418600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.419388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] RBAC: Adding query filter params (interface=public, service_id=fdf30b7bd97b427391cf03ea06ab33c3, region_id=RegionOne) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.419702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.421274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.426642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 513/1035] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/endpoints?service_id=fdf30b7bd97b427391cf03ea06ab33c3&interface=public®ion_id=RegionOne => generated 194 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:24.431801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.473123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.488388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876a77-3b41-493b-9d04-13128180f2b7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.520110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.530447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.530447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.530447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.530447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.535932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.535932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.535932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.548815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977e8d7-ff53-4af1-9a8f-da316567b82d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.552632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 523/1036] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:24.553478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.584521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.585145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.585426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.585698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.586340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.587741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.595224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 514/1037] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:24.663742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6e5fb-d171-4bf3-99a8-03f66f9c2941 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.701709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.702287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.702585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.702919 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.711273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.712955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:24.726463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:24.727544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:24.728563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:24.731583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 524/1038] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:24.733834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.843228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea7ad6-c21b-49c0-9b11-fe61295349e8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.845028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.877939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.878610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.878896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.879170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.880033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.880344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.881924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.906421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 515/1039] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:24.907763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.958939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.013716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a063b2-01df-4743-b510-9432c1af3715 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.049451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec5525-385d-442a-80de-b37f70c59c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.050193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec5525-385d-442a-80de-b37f70c59c7c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.050551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec5525-385d-442a-80de-b37f70c59c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.050896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec5525-385d-442a-80de-b37f70c59c7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.133314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec5525-385d-442a-80de-b37f70c59c7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.134863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec5525-385d-442a-80de-b37f70c59c7c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.182875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.234734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 525/1040] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:25.240556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.350075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a4ba9-98aa-412c-b712-f5906d271cd4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.385335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.386700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.387083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.387499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.398967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.400625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.416222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031009c-7f73-4c10-8f51-1bf0d3159a92 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.416494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031009c-7f73-4c10-8f51-1bf0d3159a92 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:25.416722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:25.417797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:25.418911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:25.418911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:25.418911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:25.418911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:25.419182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031009c-7f73-4c10-8f51-1bf0d3159a92 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.419182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 526/1041] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:25.422215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 516/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:25.423137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc14c-8639-43e7-ae0a-738f51216067 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.423457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc14c-8639-43e7-ae0a-738f51216067 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.423740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc14c-8639-43e7-ae0a-738f51216067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.425730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.442422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-663dc14c-8639-43e7-ae0a-738f51216067 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:25.443755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-663dc14c-8639-43e7-ae0a-738f51216067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:25.444393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.531236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 527/1043] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:25.550464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dc14c-8639-43e7-ae0a-738f51216067 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.552080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc612fd1-f8fc-4115-9eef-913a98bc056d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.587428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.588046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.588333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.588609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.589890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.590201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.591606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.605195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 517/1044] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:25.610618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.633214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.655522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dc14c-8639-43e7-ae0a-738f51216067 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.689600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.692972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.693630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.693916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.694196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.705679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:25.710527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:25.711619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:25.712625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 528/1045] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:25.720307 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.724256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab579e0-830e-4414-9614-77277b30d461 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.764537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.765190 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.765480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.765760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.773425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.774885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:25.787915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:25.788843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:25.789799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 518/1046] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:25.795269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.843855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938259e-78d4-4794-bf90-a49dd96639c1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.884359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.884984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.885269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.885546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.886770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.887079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.888530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.894459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.898476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 529/1047] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:25.904530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.910314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff365e-8dd5-4231-8606-75dffc0b2249 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.939333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.944553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.945179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.945469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.945748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.946498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:25.946809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.948855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.960811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 519/1048] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:25.967259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.015689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77833c-7aa9-4d1c-808b-5f4d11f5f278 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.060337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.060909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.061199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.061482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] PATH_INFO: `/v3/users/3a2bf3e4009840618ccd752105f8f584` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.077190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=3a2bf3e4009840618ccd752105f8f584)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.078684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.080635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 530/1049] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/users/3a2bf3e4009840618ccd752105f8f584 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:26.085011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed6da-71f6-46f3-a5b4-c8f6fbe54e0b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.086302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.126235 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.127126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.127587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.127860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.128649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] RBAC: Adding query filter params (role.id=e14505fd8ab54e46b1b6f3e87b003e87, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=9072775865604c73ae8361a047945464) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:26.128957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.130438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.137140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 520/1050] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/role_assignments?user.id=9072775865604c73ae8361a047945464&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=e14505fd8ab54e46b1b6f3e87b003e87 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:26.161635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.208754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b242b3-3637-4758-acba-653556951ed6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.246718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.247789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.248074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.248353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.328245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.329599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.388361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.432463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 531/1051] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:02:26.438193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.500102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.551851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf7e0-c328-4078-b512-bbd25defda63 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.590363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c203d-69ce-43c4-98b2-a42b0221c65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.591429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c203d-69ce-43c4-98b2-a42b0221c65a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.591919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c203d-69ce-43c4-98b2-a42b0221c65a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.592422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c203d-69ce-43c4-98b2-a42b0221c65a None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.593870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c203d-69ce-43c4-98b2-a42b0221c65a None admin] RBAC: Authorizing `identity:get_project(project_id=40fc172a073247eb842725c9f0a1ec65)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.595680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c203d-69ce-43c4-98b2-a42b0221c65a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.597627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 521/1052] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:26.604541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.685334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682cad-928b-4356-bfdd-bfa83cf17e8c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.685660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682cad-928b-4356-bfdd-bfa83cf17e8c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.685904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682cad-928b-4356-bfdd-bfa83cf17e8c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.687491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 522/1053] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:26.692130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533da34-11dc-407e-a026-fe3a0c8013da None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.692424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533da34-11dc-407e-a026-fe3a0c8013da None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.692709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533da34-11dc-407e-a026-fe3a0c8013da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.711208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5533da34-11dc-407e-a026-fe3a0c8013da None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:26.712412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5533da34-11dc-407e-a026-fe3a0c8013da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:26.717873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72641bd5-ccfe-425e-98f1-901998c75985 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.758719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.759499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.759787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.760063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.760853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=40fc172a073247eb842725c9f0a1ec65, user.id=3a2bf3e4009840618ccd752105f8f584) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:26.761226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.762710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.771537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 532/1054] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/role_assignments?user.id=3a2bf3e4009840618ccd752105f8f584&scope.project.id=40fc172a073247eb842725c9f0a1ec65&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:26.786824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 523/1055] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:26.810313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.839687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.923533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643c472-67a7-4357-b50d-88df4014ff77 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.958733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.959415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.959701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.959982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] PATH_INFO: `/v3/services/image` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.960607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.961973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:26.971430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:26.972485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:02:26.973943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:26.974996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 533/1056] 10.222.0.49 () {56 vars in 1105 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:26.978767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533da34-11dc-407e-a026-fe3a0c8013da None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.036010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.081280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533da34-11dc-407e-a026-fe3a0c8013da None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.117247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.117876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.118160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.118435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.119157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] RBAC: Adding query filter params (name=image) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:27.119515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.120986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.130924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 524/1057] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:27.131929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.222243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.237617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11f3df-b7d2-4773-a4b1-4354ecea1d65 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.273534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda21990-f121-45b6-9da1-2c81924b799d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.274160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21990-f121-45b6-9da1-2c81924b799d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.274446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21990-f121-45b6-9da1-2c81924b799d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.274723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21990-f121-45b6-9da1-2c81924b799d None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.275463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda21990-f121-45b6-9da1-2c81924b799d None admin] RBAC: Adding query filter params (type=image) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:27.275781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda21990-f121-45b6-9da1-2c81924b799d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.277783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda21990-f121-45b6-9da1-2c81924b799d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.287574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 534/1058] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:27.289066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:27.395007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c8d8a-5f40-4b6c-b418-3d8c1f0b5fce None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:27.413273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:27.429964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aa74e-3fc1-4139-8836-6dd98d450b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:27.430586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa74e-3fc1-4139-8836-6dd98d450b5e None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:27.430872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa74e-3fc1-4139-8836-6dd98d450b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:27.431148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa74e-3fc1-4139-8836-6dd98d450b5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:27.431892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6aa74e-3fc1-4139-8836-6dd98d450b5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:27.433293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6aa74e-3fc1-4139-8836-6dd98d450b5e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:27.524799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 525/1059] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:28.183549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58155de-4f51-41c4-9eab-4ee63ff6d926 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.184105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58155de-4f51-41c4-9eab-4ee63ff6d926 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.184105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58155de-4f51-41c4-9eab-4ee63ff6d926 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.185507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 535/1060] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:28.190666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038de679-54c0-4c07-a755-f10197caf8bb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.191073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038de679-54c0-4c07-a755-f10197caf8bb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.191348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038de679-54c0-4c07-a755-f10197caf8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.210477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-038de679-54c0-4c07-a755-f10197caf8bb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:28.211750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-038de679-54c0-4c07-a755-f10197caf8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:28.298175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 526/1061] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:28.318961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58155de-4f51-41c4-9eab-4ee63ff6d926 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.422303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58155de-4f51-41c4-9eab-4ee63ff6d926 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.456234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.456793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.457067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.457342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.468198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:28.472968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:28.474102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:28.475139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 536/1062] 10.222.0.49 () {56 vars in 1120 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:28.481576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038de679-54c0-4c07-a755-f10197caf8bb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.590290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:28.593916 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038de679-54c0-4c07-a755-f10197caf8bb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.630188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c124b-c5ba-4a07-b25e-8b738504eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.630812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c124b-c5ba-4a07-b25e-8b738504eee6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.631095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c124b-c5ba-4a07-b25e-8b738504eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.631398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c124b-c5ba-4a07-b25e-8b738504eee6 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.632123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c124b-c5ba-4a07-b25e-8b738504eee6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:28.632425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c124b-c5ba-4a07-b25e-8b738504eee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.633801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c124b-c5ba-4a07-b25e-8b738504eee6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.648161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 527/1063] 10.222.0.49 () {56 vars in 1115 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:28.650009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.664335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:28.765651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786e58-2c44-41ce-9d5c-81f0a661d8b9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.782296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5bdac-4e13-495f-8c51-c2953f3a1a05 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.782596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5bdac-4e13-495f-8c51-c2953f3a1a05 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.782881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5bdac-4e13-495f-8c51-c2953f3a1a05 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.784288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 528/1064] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:28.789086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1abccd-346f-4c83-afa4-c427d76134de None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.789367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1abccd-346f-4c83-afa4-c427d76134de None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.789637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1abccd-346f-4c83-afa4-c427d76134de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.803922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.804881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.805169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.805474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.808022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ac1abccd-346f-4c83-afa4-c427d76134de None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:28.809703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ac1abccd-346f-4c83-afa4-c427d76134de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:28.818751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.821040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:28.837154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:28.838244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:28.839132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:28.839132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:28.839132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:28.839132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:28.839132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 537/1065] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:28.844408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.885211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 529/1066] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:28.907889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1abccd-346f-4c83-afa4-c427d76134de None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:28.909046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:28.952904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16a765-d457-4ca6-93cd-128ca8abbed6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:28.986669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:28.987422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.987765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.988082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.988769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:28.989076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:28.990443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.007083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 538/1067] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:29.008241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.019028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.030320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1abccd-346f-4c83-afa4-c427d76134de None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.068659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.069311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.069594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.069913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.082227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:29.087931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:29.089059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:29.090062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 530/1068] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:29.095065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.124868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2beb6f-33d3-4677-a778-07536e8f5f68 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.154630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.167044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.167744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.168028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.169697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.175942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.177367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:29.191311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:29.192894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:29.193906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 539/1069] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:29.198387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.218580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e9d63-e2d2-4b88-b7ee-823396b29eac None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.256596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.257220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.257504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.257776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.258494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:29.258799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.260281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.271365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 531/1070] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:29.277265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.294638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.312522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c6fff-dc5b-4c58-a307-3abdff9e28f5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.349178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.349813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.350099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.350371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.351161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:29.351516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.352937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.376573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 540/1071] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:29.378341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.397734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b81014-3345-4ce6-a8e4-276374293130 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.424700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.441781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.442556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.442952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.443165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] PATH_INFO: `/v3/users/3a2bf3e4009840618ccd752105f8f584` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.444446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] RBAC: Authorizing `identity:get_user(user_id=3a2bf3e4009840618ccd752105f8f584)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.445770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.447727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 532/1072] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/users/3a2bf3e4009840618ccd752105f8f584 => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:29.453910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.496111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ea157-4d14-4f46-b6fc-a4f4bbbf6b95 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.542249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.542943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.543259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.543542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.552069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.555344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.556801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.571664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a09d17-4de0-4fc6-954f-eae4219a4666 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:29.572714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:29.574131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:29.575049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:29.575049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:29.575049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:29.575049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:29.575049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 541/1073] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:29.580420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.610857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731bf5e-d986-4356-a2a5-1ccfabf4c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.611543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731bf5e-d986-4356-a2a5-1ccfabf4c8bd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.611828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731bf5e-d986-4356-a2a5-1ccfabf4c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.612126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731bf5e-d986-4356-a2a5-1ccfabf4c8bd None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.613266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731bf5e-d986-4356-a2a5-1ccfabf4c8bd None admin] RBAC: Authorizing `identity:get_project(project_id=40fc172a073247eb842725c9f0a1ec65)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.614709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731bf5e-d986-4356-a2a5-1ccfabf4c8bd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.616427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 533/1074] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:29.616427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22947ec4-abd6-4e67-bb4d-9ecf1e42de27 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.616427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22947ec4-abd6-4e67-bb4d-9ecf1e42de27 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.616427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22947ec4-abd6-4e67-bb4d-9ecf1e42de27 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.616427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 534/1075] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:29.622641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22947ec4-abd6-4e67-bb4d-9ecf1e42de27 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.675607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.684891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f314db-2b87-4ca7-b0c5-3e9a386c0476 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.720713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa1072-cb36-4595-81c6-d2f448d9ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.721341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1072-cb36-4595-81c6-d2f448d9ab31 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.721627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1072-cb36-4595-81c6-d2f448d9ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.721910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1072-cb36-4595-81c6-d2f448d9ab31 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.722605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa1072-cb36-4595-81c6-d2f448d9ab31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:29.722917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa1072-cb36-4595-81c6-d2f448d9ab31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.724364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa1072-cb36-4595-81c6-d2f448d9ab31 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.725080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22947ec4-abd6-4e67-bb4d-9ecf1e42de27 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.736617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 542/1076] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:29.737607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67dc38-7bba-4816-bb8b-045bf91679b2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.738055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67dc38-7bba-4816-bb8b-045bf91679b2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.738266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67dc38-7bba-4816-bb8b-045bf91679b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.756212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6b67dc38-7bba-4816-bb8b-045bf91679b2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:29.757418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6b67dc38-7bba-4816-bb8b-045bf91679b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:29.759904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.760576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.760865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.761694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65/users/3a2bf3e4009840618ccd752105f8f584/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.763669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] RBAC: Authorizing `identity:create_grant(project_id=40fc172a073247eb842725c9f0a1ec65, user_id=3a2bf3e4009840618ccd752105f8f584, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:29.765112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:29.825153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 543/1077] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:29.826194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67dc38-7bba-4816-bb8b-045bf91679b2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.899454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 535/1078] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:29 2022] PUT /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65/users/3a2bf3e4009840618ccd752105f8f584/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:29.900291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:29.913313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.917386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:29.940834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67dc38-7bba-4816-bb8b-045bf91679b2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:29.984597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:29.985747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:29.986039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:29.986320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:29.998419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.000063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.012129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcce28c-05a7-4bb9-985a-1c852c3b159e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:30.012987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:30.013965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:30.014932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 544/1079] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:30.020698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.061697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.063258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.063505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.063784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.132321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59173b34-32dd-4eec-a575-10dfd4e913cf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.141352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.143417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.156456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.170398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.171033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.171372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.171654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.172406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:30.172719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.174209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.186676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 545/1080] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:30.192633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.195742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.263809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 536/1081] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:30.264897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.307508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699bc9b-b41f-4e2c-99c4-471ef676aa4d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.345156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.345846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.346151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.346525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/9072775865604c73ae8361a047945464/roles/e14505fd8ab54e46b1b6f3e87b003e87` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.349135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=9072775865604c73ae8361a047945464, role_id=e14505fd8ab54e46b1b6f3e87b003e87)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.350691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.368939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b966f-0166-4c29-9c98-bc522e91e945 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.380301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.404204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a413656-f19f-4d0f-b4be-f9590a21fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.404780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a413656-f19f-4d0f-b4be-f9590a21fc7f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.405064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a413656-f19f-4d0f-b4be-f9590a21fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.405340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a413656-f19f-4d0f-b4be-f9590a21fc7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.415206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a413656-f19f-4d0f-b4be-f9590a21fc7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.416666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a413656-f19f-4d0f-b4be-f9590a21fc7f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2a413656-f19f-4d0f-b4be-f9590a21fc7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:30.432404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:30.444011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:30.445118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:30.445118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:30.445118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:30.445118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:30.445387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 546/1082] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:30 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/9072775865604c73ae8361a047945464/roles/e14505fd8ab54e46b1b6f3e87b003e87 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:30.445482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 537/1083] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:30.445565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.520593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.571032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f107-117f-48df-9a43-8bf388284319 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.609811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20039b26-5be5-43e5-826b-a8bdf6a4d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.610455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20039b26-5be5-43e5-826b-a8bdf6a4d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.610743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20039b26-5be5-43e5-826b-a8bdf6a4d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.611025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20039b26-5be5-43e5-826b-a8bdf6a4d3a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.611811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20039b26-5be5-43e5-826b-a8bdf6a4d3a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:30.612124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20039b26-5be5-43e5-826b-a8bdf6a4d3a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.613512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20039b26-5be5-43e5-826b-a8bdf6a4d3a7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.627419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 547/1084] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:31.915155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd8094-5d5e-4524-a9fb-7eeba1d02077 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.915780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd8094-5d5e-4524-a9fb-7eeba1d02077 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.915780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd8094-5d5e-4524-a9fb-7eeba1d02077 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.916999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 538/1085] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:31.922171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae239-268d-479e-8016-85f08c28ca27 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.922584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae239-268d-479e-8016-85f08c28ca27 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.922865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae239-268d-479e-8016-85f08c28ca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.940650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3f8ae239-268d-479e-8016-85f08c28ca27 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:31.941839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3f8ae239-268d-479e-8016-85f08c28ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:32.016638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 548/1086] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:32.039671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd8094-5d5e-4524-a9fb-7eeba1d02077 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.143276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd8094-5d5e-4524-a9fb-7eeba1d02077 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.181668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.182223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.182510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.182788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.193975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:32.198783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:32.200437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:32.201301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 539/1087] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:32.206725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae239-268d-479e-8016-85f08c28ca27 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.260048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.305214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae239-268d-479e-8016-85f08c28ca27 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.340328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef75dd0-0c65-4f3b-8ebb-74bfaf000755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.340961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75dd0-0c65-4f3b-8ebb-74bfaf000755 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.341240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75dd0-0c65-4f3b-8ebb-74bfaf000755 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.341514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75dd0-0c65-4f3b-8ebb-74bfaf000755 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.342224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef75dd0-0c65-4f3b-8ebb-74bfaf000755 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:32.342530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef75dd0-0c65-4f3b-8ebb-74bfaf000755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.343942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef75dd0-0c65-4f3b-8ebb-74bfaf000755 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.352159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 549/1088] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:32.357914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.383497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.455320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85413e1-02d6-4393-af24-e7938870dbe3 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.455659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85413e1-02d6-4393-af24-e7938870dbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.455951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85413e1-02d6-4393-af24-e7938870dbe3 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.457345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 550/1089] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:32.458640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e9418-45fa-43dd-a09c-ddee76074545 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.462199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbccd8f-b0b7-4b00-a683-66c932de3499 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.462461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbccd8f-b0b7-4b00-a683-66c932de3499 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.462743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbccd8f-b0b7-4b00-a683-66c932de3499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.480967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dcbccd8f-b0b7-4b00-a683-66c932de3499 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:32.482153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dcbccd8f-b0b7-4b00-a683-66c932de3499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:32.493143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.493788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.494073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.494352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] PATH_INFO: `/v3/users/3a2bf3e4009840618ccd752105f8f584` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.495551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] RBAC: Authorizing `identity:get_user(user_id=3a2bf3e4009840618ccd752105f8f584)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.497853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.499494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 540/1090] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/users/3a2bf3e4009840618ccd752105f8f584 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:32.505115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.552314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 551/1091] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:02:32.576266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbccd8f-b0b7-4b00-a683-66c932de3499 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.604072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.613899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca113d33-331e-4bee-97dd-464e61d683db None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.623417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.647974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.648606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.648895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.649231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.675758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbccd8f-b0b7-4b00-a683-66c932de3499 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.710571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.711178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.711490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.711779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.718176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.719795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.721953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:32.726841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:32.727809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:32.728639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 552/1092] 10.222.0.49 () {56 vars in 1120 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:32.734571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.813062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.823094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 541/1093] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:32.829002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.832772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2531a-0bd2-4410-a6d1-99ec3a049523 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.835203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.867869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.868490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.868774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.869054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.869772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:32.870075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.871481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.880329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 553/1094] 10.222.0.49 () {56 vars in 1115 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:32.885890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.929128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc21c22-a89b-4d92-a631-ac70b5c96638 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.963710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.964285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.965121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.965404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.966569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] RBAC: Authorizing `identity:get_project(project_id=40fc172a073247eb842725c9f0a1ec65)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.968048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.969755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 542/1095] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:32.975849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.986910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860d38-2ba8-4dc5-bca4-aeab0f2cbc93 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.020746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.021391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.021683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.021960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.032926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.034364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.034933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:33.049084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:33.050710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:33.051848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:33.051848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:33.051848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:33.051848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:33.051848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 554/1096] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:02:33.055939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.075492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.079597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d6908-7798-45bd-8488-a0e3b7c163d7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.114095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.114867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.115290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.115578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.116371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=40fc172a073247eb842725c9f0a1ec65, user.id=3a2bf3e4009840618ccd752105f8f584) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:33.116686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.118130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.124258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 543/1097] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:32 2022] GET /identity/v3/role_assignments?user.id=3a2bf3e4009840618ccd752105f8f584&scope.project.id=40fc172a073247eb842725c9f0a1ec65&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:33.158207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aeafaa-4312-4217-9b90-d49ff786e71e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.194702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39917e-6275-44da-918b-99088a74423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.195482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39917e-6275-44da-918b-99088a74423e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.195840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39917e-6275-44da-918b-99088a74423e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.196127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39917e-6275-44da-918b-99088a74423e None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.196843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39917e-6275-44da-918b-99088a74423e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:33.197156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39917e-6275-44da-918b-99088a74423e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.198584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39917e-6275-44da-918b-99088a74423e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.212812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 555/1098] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:33.219167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.332005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891aff-cc0e-4001-af78-90691c9c713b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.355013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.366675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.367371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.367660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.367949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.376180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.377743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:02:33.390603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:33.408283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:33.409485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 544/1099] 10.222.0.49 () {56 vars in 1093 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:33.410243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39917e-6275-44da-918b-99088a74423e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.483604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.505101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39917e-6275-44da-918b-99088a74423e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.542852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.543907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.544405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.544906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.546378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:33.546940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.550053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.577086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 556/1100] 10.222.0.49 () {56 vars in 1133 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:33.582967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.686787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3460c-7665-4696-989f-35d13f1af5a9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.720508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.721128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.721417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.721699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.771392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.789790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.791341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.892679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 545/1101] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:33.898504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.000264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8b964-ca09-4b8f-8c07-88bf3964d3f8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.034686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.035363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.035710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.036025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.038755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.047201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.048719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:02:34.063667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:34.064582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:34.065473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:02:34.065473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:02:34.065473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:02:34.065473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:34.065473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 557/1102] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:34.070508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.150365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.168723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c560a6-5d48-4ef7-a4ca-ec8598f14574 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.207006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.207666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.208051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.208442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.209148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.209453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.210854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.224270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 546/1103] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:34.229804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.333644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf269bd-4a3a-41b3-9434-90714c6aa8bf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.367411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.368058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.368342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.368618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.376172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.377602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:02:34.389320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:34.390459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:34.392150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 558/1104] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:34.396403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.503049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904bfe1-1196-43e9-8068-38d461304ef1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.540191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa88c9-a647-43c4-bfff-94c44cb878be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.540814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa88c9-a647-43c4-bfff-94c44cb878be None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.541100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa88c9-a647-43c4-bfff-94c44cb878be None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.541376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa88c9-a647-43c4-bfff-94c44cb878be None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.542125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa88c9-a647-43c4-bfff-94c44cb878be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.542436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa88c9-a647-43c4-bfff-94c44cb878be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.544525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa88c9-a647-43c4-bfff-94c44cb878be None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.556538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 547/1105] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:34.562642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.612762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.664257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa1ab3-8535-49e1-b078-5aeade82c9b8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.697240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b437d84-3a85-47a6-8185-2d68979a6a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.697848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437d84-3a85-47a6-8185-2d68979a6a85 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.698134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437d84-3a85-47a6-8185-2d68979a6a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.698413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437d84-3a85-47a6-8185-2d68979a6a85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.699758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b437d84-3a85-47a6-8185-2d68979a6a85 None admin] RBAC: Adding query filter params (role.id=e14505fd8ab54e46b1b6f3e87b003e87, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=9072775865604c73ae8361a047945464) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:34.700069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b437d84-3a85-47a6-8185-2d68979a6a85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.701495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b437d84-3a85-47a6-8185-2d68979a6a85 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.707242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 559/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/role_assignments?user.id=9072775865604c73ae8361a047945464&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=e14505fd8ab54e46b1b6f3e87b003e87 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:35.181835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef542-0908-4b1e-bd6f-31af92d172c9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.182351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef542-0908-4b1e-bd6f-31af92d172c9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.182430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef542-0908-4b1e-bd6f-31af92d172c9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.184318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 548/1107] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:35.189767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9f18f-10d9-4d6f-b6ee-71833c4a16ae None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.190199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9f18f-10d9-4d6f-b6ee-71833c4a16ae None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.190339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9f18f-10d9-4d6f-b6ee-71833c4a16ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.208653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bed9f18f-10d9-4d6f-b6ee-71833c4a16ae None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:35.209854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bed9f18f-10d9-4d6f-b6ee-71833c4a16ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:35.279218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 560/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:35.304425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ef542-0908-4b1e-bd6f-31af92d172c9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.307486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.412084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ef542-0908-4b1e-bd6f-31af92d172c9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.447262 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951543-7891-40c8-a38a-665357627410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.447937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951543-7891-40c8-a38a-665357627410 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.448223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951543-7891-40c8-a38a-665357627410 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.448517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951543-7891-40c8-a38a-665357627410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.450177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951543-7891-40c8-a38a-665357627410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.451716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951543-7891-40c8-a38a-665357627410 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.453577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 549/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:35.459420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9f18f-10d9-4d6f-b6ee-71833c4a16ae None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.504058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.564102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9f18f-10d9-4d6f-b6ee-71833c4a16ae None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.598388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfb0f5-7462-4ff0-96ec-c82485b87246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.599007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb0f5-7462-4ff0-96ec-c82485b87246 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.599329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb0f5-7462-4ff0-96ec-c82485b87246 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.599612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb0f5-7462-4ff0-96ec-c82485b87246 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.600300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfb0f5-7462-4ff0-96ec-c82485b87246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.601727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfb0f5-7462-4ff0-96ec-c82485b87246 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.719345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 561/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:35 2022] POST /identity/v3/users => generated 286 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:37.749353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875735d-2ef4-42b5-a5ee-bca832d5f373 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.749927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875735d-2ef4-42b5-a5ee-bca832d5f373 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.749927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875735d-2ef4-42b5-a5ee-bca832d5f373 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.751262 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 550/1111] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:37.756280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004a94-f9f8-4687-bad9-097012952464 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.756716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004a94-f9f8-4687-bad9-097012952464 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.756832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004a94-f9f8-4687-bad9-097012952464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.775021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ff004a94-f9f8-4687-bad9-097012952464 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:37.776217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ff004a94-f9f8-4687-bad9-097012952464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:37.842766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 562/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:37.865698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875735d-2ef4-42b5-a5ee-bca832d5f373 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.939293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:37.971287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875735d-2ef4-42b5-a5ee-bca832d5f373 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.009745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.010511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.010939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.011425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] PATH_INFO: `/v3/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.013497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] RBAC: Authorizing `identity:get_role(role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.015525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.017420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 551/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/roles/9a49213a53964452a411e2dbd74d7c5e => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:38.023355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004a94-f9f8-4687-bad9-097012952464 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.051505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.121023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004a94-f9f8-4687-bad9-097012952464 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.154552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.155230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.155550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.156056 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] PATH_INFO: `/v3/users/3fcdd04ebe0749e490dcf3505bcc8d3f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.170542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] RBAC: Authorizing `identity:get_user(user_id=3fcdd04ebe0749e490dcf3505bcc8d3f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.172020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.173664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 563/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/users/3fcdd04ebe0749e490dcf3505bcc8d3f => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:38.179619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.286855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5452692-ab82-400d-b42f-6c7c61ac0e06 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.319609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.320221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.320501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.320777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.329159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.386859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.388259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.482910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 552/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:38.489367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.573675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.588324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad123d-d03b-4354-a80b-dba01030357a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.624229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659a84f-525d-4f0c-b1fe-01ea21b420c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.624888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a84f-525d-4f0c-b1fe-01ea21b420c2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.625174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a84f-525d-4f0c-b1fe-01ea21b420c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.625456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a84f-525d-4f0c-b1fe-01ea21b420c2 None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.626619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659a84f-525d-4f0c-b1fe-01ea21b420c2 None admin] RBAC: Authorizing `identity:get_project(project_id=40fc172a073247eb842725c9f0a1ec65)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.628089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659a84f-525d-4f0c-b1fe-01ea21b420c2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.629773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 564/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:38.635962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:38.674198 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:38.731525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3584a9-88e7-45dd-b5d6-12504a3a440c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:38.764254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3718c1d-f32b-49e6-833b-6b7305e28f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:38.764858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3718c1d-f32b-49e6-833b-6b7305e28f22 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:38.765138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3718c1d-f32b-49e6-833b-6b7305e28f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:38.765414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3718c1d-f32b-49e6-833b-6b7305e28f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:38.766193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3718c1d-f32b-49e6-833b-6b7305e28f22 None admin] RBAC: Adding query filter params (role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=40fc172a073247eb842725c9f0a1ec65, user.id=3fcdd04ebe0749e490dcf3505bcc8d3f) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:38.766498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3718c1d-f32b-49e6-833b-6b7305e28f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:38.767939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3718c1d-f32b-49e6-833b-6b7305e28f22 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:38.773179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 553/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:38 2022] GET /identity/v3/role_assignments?user.id=3fcdd04ebe0749e490dcf3505bcc8d3f&scope.project.id=40fc172a073247eb842725c9f0a1ec65&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:40.780063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd95aaa-a26c-44ac-8824-ea8773941e3e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.780592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd95aaa-a26c-44ac-8824-ea8773941e3e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.780592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd95aaa-a26c-44ac-8824-ea8773941e3e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.782025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 565/1118] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:02:40.787351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43814d8-3c8a-4c07-bb52-58244a2aa1ed None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.787830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43814d8-3c8a-4c07-bb52-58244a2aa1ed None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.787941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43814d8-3c8a-4c07-bb52-58244a2aa1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:40.806170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e43814d8-3c8a-4c07-bb52-58244a2aa1ed None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:40.807405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e43814d8-3c8a-4c07-bb52-58244a2aa1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:40.876673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 554/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:40.899327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd95aaa-a26c-44ac-8824-ea8773941e3e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.003148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd95aaa-a26c-44ac-8824-ea8773941e3e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.036726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.037299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.037635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.037913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] PATH_INFO: `/v3/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.039383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] RBAC: Authorizing `identity:get_role(role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.040744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.042090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 566/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:40 2022] GET /identity/v3/roles/9a49213a53964452a411e2dbd74d7c5e => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:41.048146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43814d8-3c8a-4c07-bb52-58244a2aa1ed None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.079060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:41.145641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43814d8-3c8a-4c07-bb52-58244a2aa1ed None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.179168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.179833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.180117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.180391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] PATH_INFO: `/v3/users/3fcdd04ebe0749e490dcf3505bcc8d3f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.181622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcdd04ebe0749e490dcf3505bcc8d3f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.183051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.185199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 555/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:41 2022] GET /identity/v3/users/3fcdd04ebe0749e490dcf3505bcc8d3f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:41.190949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.218279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:41.297419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048549c8-f2dc-4bba-9a28-5d0107bcd9ed None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.330615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763506e-d98a-4d47-b984-239db95efc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.331276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763506e-d98a-4d47-b984-239db95efc0d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.331567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763506e-d98a-4d47-b984-239db95efc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.331842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763506e-d98a-4d47-b984-239db95efc0d None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.332974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763506e-d98a-4d47-b984-239db95efc0d None admin] RBAC: Authorizing `identity:get_project(project_id=40fc172a073247eb842725c9f0a1ec65)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.334403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763506e-d98a-4d47-b984-239db95efc0d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.336097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 567/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:41 2022] GET /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:41.342175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:41.437712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:41.440429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7712-1430-4265-b82a-6a9babfd0d82 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:41.473744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400324-f5bb-4a51-b59c-d0b03210a60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:41.474417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400324-f5bb-4a51-b59c-d0b03210a60c None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:41.474743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400324-f5bb-4a51-b59c-d0b03210a60c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:41.475022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400324-f5bb-4a51-b59c-d0b03210a60c None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65/users/3fcdd04ebe0749e490dcf3505bcc8d3f/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.477214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400324-f5bb-4a51-b59c-d0b03210a60c None admin] RBAC: Authorizing `identity:create_grant(project_id=40fc172a073247eb842725c9f0a1ec65, user_id=3fcdd04ebe0749e490dcf3505bcc8d3f, role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:41.478657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400324-f5bb-4a51-b59c-d0b03210a60c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:41.553536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 556/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:41 2022] PUT /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65/users/3fcdd04ebe0749e490dcf3505bcc8d3f/roles/9a49213a53964452a411e2dbd74d7c5e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:43.527378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90008e4-76f4-4aaf-b524-b2eeec858457 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.528477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90008e4-76f4-4aaf-b524-b2eeec858457 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.528477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90008e4-76f4-4aaf-b524-b2eeec858457 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.529798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 568/1124] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:43.535117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daaae4-04f6-4a2d-8869-2fe1cb578caa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.535563 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daaae4-04f6-4a2d-8869-2fe1cb578caa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.535692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daaae4-04f6-4a2d-8869-2fe1cb578caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.553921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-97daaae4-04f6-4a2d-8869-2fe1cb578caa None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:43.555109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-97daaae4-04f6-4a2d-8869-2fe1cb578caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:43.624821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 557/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:43.648690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90008e4-76f4-4aaf-b524-b2eeec858457 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:43.739451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:43.750410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90008e4-76f4-4aaf-b524-b2eeec858457 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:43.788154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:43.788828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.789213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.789598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] PATH_INFO: `/v3/roles/9a49213a53964452a411e2dbd74d7c5e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.791475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] RBAC: Authorizing `identity:get_role(role_id=9a49213a53964452a411e2dbd74d7c5e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:43.794604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:43.796609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 569/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:43 2022] GET /identity/v3/roles/9a49213a53964452a411e2dbd74d7c5e => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:43.802638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daaae4-04f6-4a2d-8869-2fe1cb578caa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:43.890645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:43.899784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daaae4-04f6-4a2d-8869-2fe1cb578caa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:43.933954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:43.935144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:43.935457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:43.935731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] PATH_INFO: `/v3/users/3fcdd04ebe0749e490dcf3505bcc8d3f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:43.936879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcdd04ebe0749e490dcf3505bcc8d3f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:43.938308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:43.939924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 558/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:43 2022] GET /identity/v3/users/3fcdd04ebe0749e490dcf3505bcc8d3f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:43.945768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:44.048203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c703f8-95ac-4c68-b7bf-0584b08daab0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:44.083314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:44.083926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.084214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.084493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.110657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:44.151504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:44.153511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:44.249916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 570/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:44.256032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:44.354936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f4eb4-5941-4892-bba4-dd2cc96d6bb6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:44.388585 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d401db0-6b9c-48b4-8519-9bc69a8f56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:44.389219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401db0-6b9c-48b4-8519-9bc69a8f56b9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.389501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401db0-6b9c-48b4-8519-9bc69a8f56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.389777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401db0-6b9c-48b4-8519-9bc69a8f56b9 None admin] PATH_INFO: `/v3/projects/40fc172a073247eb842725c9f0a1ec65` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.390929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d401db0-6b9c-48b4-8519-9bc69a8f56b9 None admin] RBAC: Authorizing `identity:get_project(project_id=40fc172a073247eb842725c9f0a1ec65)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:44.392400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d401db0-6b9c-48b4-8519-9bc69a8f56b9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:44.394209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 559/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:44 2022] GET /identity/v3/projects/40fc172a073247eb842725c9f0a1ec65 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:44.400445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:44.458196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:44.500127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc850ed-344c-432e-90e0-064e9c92f297 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:44.535014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745efce-3b1c-44c8-a451-6beb741f89d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:44.535669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745efce-3b1c-44c8-a451-6beb741f89d0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:44.535952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745efce-3b1c-44c8-a451-6beb741f89d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:44.536231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745efce-3b1c-44c8-a451-6beb741f89d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:44.537014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745efce-3b1c-44c8-a451-6beb741f89d0 None admin] RBAC: Adding query filter params (role.id=9a49213a53964452a411e2dbd74d7c5e, scope.project.id=40fc172a073247eb842725c9f0a1ec65, user.id=3fcdd04ebe0749e490dcf3505bcc8d3f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:44.537329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745efce-3b1c-44c8-a451-6beb741f89d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:44.538793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745efce-3b1c-44c8-a451-6beb741f89d0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:44.544828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 571/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:44 2022] GET /identity/v3/role_assignments?user.id=3fcdd04ebe0749e490dcf3505bcc8d3f&scope.project.id=40fc172a073247eb842725c9f0a1ec65&role.id=9a49213a53964452a411e2dbd74d7c5e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:46.573783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa228e-dcd0-495c-9cc7-68e392e551e2 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.574289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa228e-dcd0-495c-9cc7-68e392e551e2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.574380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa228e-dcd0-495c-9cc7-68e392e551e2 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.576359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 560/1131] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:46.581447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82f803-ef0f-452e-817c-dd3147715f01 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.581832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82f803-ef0f-452e-817c-dd3147715f01 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.582006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82f803-ef0f-452e-817c-dd3147715f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.599814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fd82f803-ef0f-452e-817c-dd3147715f01 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:46.600989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fd82f803-ef0f-452e-817c-dd3147715f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:46.667110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 572/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:46.690629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa228e-dcd0-495c-9cc7-68e392e551e2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.718644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.794255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa228e-dcd0-495c-9cc7-68e392e551e2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.828297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a0358-ce94-465d-8174-73e2a73ccc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.828940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a0358-ce94-465d-8174-73e2a73ccc5c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.829228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a0358-ce94-465d-8174-73e2a73ccc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.829509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a0358-ce94-465d-8174-73e2a73ccc5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.830665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a0358-ce94-465d-8174-73e2a73ccc5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:46.832136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a0358-ce94-465d-8174-73e2a73ccc5c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:46.833736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 561/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:46.839741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82f803-ef0f-452e-817c-dd3147715f01 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:46.894628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:46.942310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82f803-ef0f-452e-817c-dd3147715f01 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:46.976337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da19f33-3cd3-416e-a9e7-236c6e535f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:46.976952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19f33-3cd3-416e-a9e7-236c6e535f7e None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:46.977241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19f33-3cd3-416e-a9e7-236c6e535f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:46.977516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19f33-3cd3-416e-a9e7-236c6e535f7e None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:46.978207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da19f33-3cd3-416e-a9e7-236c6e535f7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:46.979672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da19f33-3cd3-416e-a9e7-236c6e535f7e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:47.072252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 573/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:02:46 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:49.123893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129257e6-ffcd-462a-bf70-66876e2c0877 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.124695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129257e6-ffcd-462a-bf70-66876e2c0877 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.124808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129257e6-ffcd-462a-bf70-66876e2c0877 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.126072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 562/1135] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:49.131378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579d20f-2c33-4679-94fd-92f255d1357b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.131802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579d20f-2c33-4679-94fd-92f255d1357b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.131951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579d20f-2c33-4679-94fd-92f255d1357b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.150281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7579d20f-2c33-4679-94fd-92f255d1357b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:49.151520 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7579d20f-2c33-4679-94fd-92f255d1357b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:49.216553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 574/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:49.240762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129257e6-ffcd-462a-bf70-66876e2c0877 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.335866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.343389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129257e6-ffcd-462a-bf70-66876e2c0877 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.377330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02e83b-bad0-41be-96cb-d50a92fad94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.377905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02e83b-bad0-41be-96cb-d50a92fad94d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.378187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02e83b-bad0-41be-96cb-d50a92fad94d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.378458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02e83b-bad0-41be-96cb-d50a92fad94d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.379693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02e83b-bad0-41be-96cb-d50a92fad94d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.381177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02e83b-bad0-41be-96cb-d50a92fad94d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.382767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 563/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:02:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:49.392991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579d20f-2c33-4679-94fd-92f255d1357b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.425257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.488189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579d20f-2c33-4679-94fd-92f255d1357b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.522593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a22dd-519e-42c0-b011-9e29b10d2d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.523240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a22dd-519e-42c0-b011-9e29b10d2d00 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.523531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a22dd-519e-42c0-b011-9e29b10d2d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.523833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a22dd-519e-42c0-b011-9e29b10d2d00 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.524536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a22dd-519e-42c0-b011-9e29b10d2d00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.525955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a22dd-519e-42c0-b011-9e29b10d2d00 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.640859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 575/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:02:49 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:51.678722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1857978-938e-47c6-a8ad-aa6fedff6fcd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.679278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1857978-938e-47c6-a8ad-aa6fedff6fcd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.679278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1857978-938e-47c6-a8ad-aa6fedff6fcd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.680659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 564/1139] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:51.686481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731125-a41c-4c61-9348-c6d1d506ca02 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.686879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731125-a41c-4c61-9348-c6d1d506ca02 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.687044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731125-a41c-4c61-9348-c6d1d506ca02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.705971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8f731125-a41c-4c61-9348-c6d1d506ca02 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:51.707167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8f731125-a41c-4c61-9348-c6d1d506ca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:51.773139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 576/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:51.796498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1857978-938e-47c6-a8ad-aa6fedff6fcd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.897617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1857978-938e-47c6-a8ad-aa6fedff6fcd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.931896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.932534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.932822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.933103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.943407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:51.948381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:51.949529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:51.951147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 565/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:51.956247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f731125-a41c-4c61-9348-c6d1d506ca02 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.978304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.056352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f731125-a41c-4c61-9348-c6d1d506ca02 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.091672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.092292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.092580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.092861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.093583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:52.093896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.095411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.103426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 577/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:52.109363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.134856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.210806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79efa6-903c-4c19-8165-23417278ddc8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.244361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.245012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.245298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.245577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] PATH_INFO: `/v3/users/759367cbff1947e5a2926841b9e1f195` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.259799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] RBAC: Authorizing `identity:get_user(user_id=759367cbff1947e5a2926841b9e1f195)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.261352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.263589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 566/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/users/759367cbff1947e5a2926841b9e1f195 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:52.269410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.350604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.368165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcc6b0-6f24-4365-8d3a-54e673ebc207 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.402036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.402664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.402948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.403251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.471243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.472695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.557330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.575401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 578/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:52.581081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.661821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:52.680491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8ca4-c6e9-455d-99cd-01eda7dfeb1d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.714185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef838fdf-4971-487e-9621-905549380725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.714857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef838fdf-4971-487e-9621-905549380725 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.715143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef838fdf-4971-487e-9621-905549380725 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.715452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef838fdf-4971-487e-9621-905549380725 None admin] PATH_INFO: `/v3/projects/c352f9ee37604220b465524a95e65159` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.717169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef838fdf-4971-487e-9621-905549380725 None admin] RBAC: Authorizing `identity:get_project(project_id=c352f9ee37604220b465524a95e65159)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.718607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef838fdf-4971-487e-9621-905549380725 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.720300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 567/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/projects/c352f9ee37604220b465524a95e65159 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:52.726428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:52.826147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1eb4f-ae65-4547-a117-9f75697e851f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:52.859313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d415899-32a5-46a8-b5cb-c19fb4e48410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:52.859928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415899-32a5-46a8-b5cb-c19fb4e48410 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.860213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415899-32a5-46a8-b5cb-c19fb4e48410 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.860492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415899-32a5-46a8-b5cb-c19fb4e48410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.861274 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d415899-32a5-46a8-b5cb-c19fb4e48410 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=c352f9ee37604220b465524a95e65159, user.id=759367cbff1947e5a2926841b9e1f195) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:52.861589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d415899-32a5-46a8-b5cb-c19fb4e48410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:52.863045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d415899-32a5-46a8-b5cb-c19fb4e48410 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:52.868557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 579/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:52 2022] GET /identity/v3/role_assignments?user.id=759367cbff1947e5a2926841b9e1f195&scope.project.id=c352f9ee37604220b465524a95e65159&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:54.833767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547213-d76c-4e1e-9264-d83feaacb0dd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.834277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547213-d76c-4e1e-9264-d83feaacb0dd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.834277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547213-d76c-4e1e-9264-d83feaacb0dd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.835611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 568/1147] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:54.840828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c2ba-c9e8-49dc-9985-d0cad5dccfd3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.841737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c2ba-c9e8-49dc-9985-d0cad5dccfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.841885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c2ba-c9e8-49dc-9985-d0cad5dccfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.860296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fbf9c2ba-c9e8-49dc-9985-d0cad5dccfd3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:54.861478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fbf9c2ba-c9e8-49dc-9985-d0cad5dccfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:54.901907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.926316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 580/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:54.948100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547213-d76c-4e1e-9264-d83feaacb0dd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.048177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547213-d76c-4e1e-9264-d83feaacb0dd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.088237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.088795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.089078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.089352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.099165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:55.103965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:55.105047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:02:55.105894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 569/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:55.111678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c2ba-c9e8-49dc-9985-d0cad5dccfd3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.206791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c2ba-c9e8-49dc-9985-d0cad5dccfd3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.240163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.240763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.241048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.241323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.242029 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:55.242335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.243728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.252503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 581/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:55.258282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.269501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.355971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502559be-f1c7-4a0d-a389-1f42f6363f64 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.389541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.390173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.390467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.390751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] PATH_INFO: `/v3/users/759367cbff1947e5a2926841b9e1f195` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.391916 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] RBAC: Authorizing `identity:get_user(user_id=759367cbff1947e5a2926841b9e1f195)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.393346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.394934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 570/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/users/759367cbff1947e5a2926841b9e1f195 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:55.400774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.448019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.507622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0316e-53fd-4fa0-af54-4600dd6d5ae9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.541326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390db896-e86a-4031-9b22-b0d081334499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.541966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390db896-e86a-4031-9b22-b0d081334499 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.542273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390db896-e86a-4031-9b22-b0d081334499 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.542545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390db896-e86a-4031-9b22-b0d081334499 None admin] PATH_INFO: `/v3/projects/c352f9ee37604220b465524a95e65159` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.543718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390db896-e86a-4031-9b22-b0d081334499 None admin] RBAC: Authorizing `identity:get_project(project_id=c352f9ee37604220b465524a95e65159)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.545765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390db896-e86a-4031-9b22-b0d081334499 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.547469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 582/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/projects/c352f9ee37604220b465524a95e65159 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:55.553477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.632179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.654757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d26d8-3249-47fb-8455-7d3a50d24d0b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.688618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35930e3-a1ab-4b73-97c7-39764be2cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.689273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35930e3-a1ab-4b73-97c7-39764be2cd99 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.689555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35930e3-a1ab-4b73-97c7-39764be2cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.689830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35930e3-a1ab-4b73-97c7-39764be2cd99 None admin] PATH_INFO: `/v3/projects/c352f9ee37604220b465524a95e65159/users/759367cbff1947e5a2926841b9e1f195/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.691772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35930e3-a1ab-4b73-97c7-39764be2cd99 None admin] RBAC: Authorizing `identity:create_grant(project_id=c352f9ee37604220b465524a95e65159, user_id=759367cbff1947e5a2926841b9e1f195, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.693210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35930e3-a1ab-4b73-97c7-39764be2cd99 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.787792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 571/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:02:55 2022] PUT /identity/v3/projects/c352f9ee37604220b465524a95e65159/users/759367cbff1947e5a2926841b9e1f195/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:57.776358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751c03-afa4-4440-a6d0-a1585303b585 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.776891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751c03-afa4-4440-a6d0-a1585303b585 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.776891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751c03-afa4-4440-a6d0-a1585303b585 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.778254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 583/1154] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:02:57.783429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32726e87-0ced-430b-8c8c-162e3e3790cf None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:57.783802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32726e87-0ced-430b-8c8c-162e3e3790cf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:57.783945 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32726e87-0ced-430b-8c8c-162e3e3790cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:57.802301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-32726e87-0ced-430b-8c8c-162e3e3790cf None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:57.803927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-32726e87-0ced-430b-8c8c-162e3e3790cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:57.871745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 572/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:02:57.894919 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27751c03-afa4-4440-a6d0-a1585303b585 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:57.993937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27751c03-afa4-4440-a6d0-a1585303b585 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.000839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.032425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.033055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.033337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.033660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.044539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:02:58.049546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:02:58.050667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:02:58.051741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 584/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:02:58.058770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32726e87-0ced-430b-8c8c-162e3e3790cf None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.122166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.156291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32726e87-0ced-430b-8c8c-162e3e3790cf None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.189688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.190267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.190550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.190827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.191559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:58.191867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.193239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.201721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 573/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:58.207533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.305661 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08307ff6-e10d-4e3c-8681-113c64303d95 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.338476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.339172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.339420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.339705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] PATH_INFO: `/v3/users/759367cbff1947e5a2926841b9e1f195` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.340852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] RBAC: Authorizing `identity:get_user(user_id=759367cbff1947e5a2926841b9e1f195)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.342433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.344096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 585/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/users/759367cbff1947e5a2926841b9e1f195 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:58.349987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.448829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64c9e-5bb0-467c-9790-324cfaa7f94a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.475735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.483548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.484174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.487100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.487100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.552514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.553963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.650875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 574/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:02:58.656733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.690741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.754125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a321-7e8d-4824-ba65-1e194186ee75 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.785896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205ae9e-f070-4fdb-bda7-449480d7dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.786525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205ae9e-f070-4fdb-bda7-449480d7dab6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.786811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205ae9e-f070-4fdb-bda7-449480d7dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.787083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205ae9e-f070-4fdb-bda7-449480d7dab6 None admin] PATH_INFO: `/v3/projects/c352f9ee37604220b465524a95e65159` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.788283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205ae9e-f070-4fdb-bda7-449480d7dab6 None admin] RBAC: Authorizing `identity:get_project(project_id=c352f9ee37604220b465524a95e65159)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.789699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205ae9e-f070-4fdb-bda7-449480d7dab6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.791381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 586/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/projects/c352f9ee37604220b465524a95e65159 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:58.798288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.828937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:58.900631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa28d-e0b1-46f0-9dca-70db92de1057 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.935322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b607-41e9-4587-8eeb-814842ca742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.935881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b607-41e9-4587-8eeb-814842ca742e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.936169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b607-41e9-4587-8eeb-814842ca742e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.936446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b607-41e9-4587-8eeb-814842ca742e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.937243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3b607-41e9-4587-8eeb-814842ca742e None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=c352f9ee37604220b465524a95e65159, user.id=759367cbff1947e5a2926841b9e1f195) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:58.937560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3b607-41e9-4587-8eeb-814842ca742e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.938994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3b607-41e9-4587-8eeb-814842ca742e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.944776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 575/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/role_assignments?user.id=759367cbff1947e5a2926841b9e1f195&scope.project.id=c352f9ee37604220b465524a95e65159&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:00.955896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51ae6f-33dc-4d5b-a2c4-477af004f3fc None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:00.956413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51ae6f-33dc-4d5b-a2c4-477af004f3fc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:00.956413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51ae6f-33dc-4d5b-a2c4-477af004f3fc None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:00.957719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 587/1162] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:00.962931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385b363-973e-468f-873b-a3b6d5b581f6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:00.963353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385b363-973e-468f-873b-a3b6d5b581f6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:00.963534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385b363-973e-468f-873b-a3b6d5b581f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:00.982020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4385b363-973e-468f-873b-a3b6d5b581f6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:00.983233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4385b363-973e-468f-873b-a3b6d5b581f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:01.050794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 576/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:01.072221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51ae6f-33dc-4d5b-a2c4-477af004f3fc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.172105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51ae6f-33dc-4d5b-a2c4-477af004f3fc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.173609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:01.206051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:01.206607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.206895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.207174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.275430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:01.276881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.371342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 588/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:03:01.378992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385b363-973e-468f-873b-a3b6d5b581f6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.402492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:01.476332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385b363-973e-468f-873b-a3b6d5b581f6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.514613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5224d2f-c6c4-484f-8846-867e44d53f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:01.515636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5224d2f-c6c4-484f-8846-867e44d53f7c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.515928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5224d2f-c6c4-484f-8846-867e44d53f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.516206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5224d2f-c6c4-484f-8846-867e44d53f7c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.523518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5224d2f-c6c4-484f-8846-867e44d53f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:01.524973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5224d2f-c6c4-484f-8846-867e44d53f7c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d5224d2f-c6c4-484f-8846-867e44d53f7c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:01.538191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:01.539463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:03:01.540562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 577/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:01.545427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.547679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:01.645807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8c81b-9c51-41fe-9944-d431781c44f7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.677802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7ed85-0164-448b-8c7b-f7d622671595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:01.678409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7ed85-0164-448b-8c7b-f7d622671595 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.678711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7ed85-0164-448b-8c7b-f7d622671595 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.678986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7ed85-0164-448b-8c7b-f7d622671595 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.679693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7ed85-0164-448b-8c7b-f7d622671595 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:01.679999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7ed85-0164-448b-8c7b-f7d622671595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:01.681348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7ed85-0164-448b-8c7b-f7d622671595 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.688838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 589/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:03.680591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388daca-b77b-4b99-aecb-dff267199d7c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.681131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388daca-b77b-4b99-aecb-dff267199d7c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.681131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388daca-b77b-4b99-aecb-dff267199d7c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.682505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 578/1167] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:03.687703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d0bd-3413-4dc5-9d8b-d9bea4afc610 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.688129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d0bd-3413-4dc5-9d8b-d9bea4afc610 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.688261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d0bd-3413-4dc5-9d8b-d9bea4afc610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.705926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9892d0bd-3413-4dc5-9d8b-d9bea4afc610 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:03.707133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9892d0bd-3413-4dc5-9d8b-d9bea4afc610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:03.768021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:03.775319 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 590/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:03.797545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388daca-b77b-4b99-aecb-dff267199d7c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.879125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:03.898048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388daca-b77b-4b99-aecb-dff267199d7c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.931755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780addb-921b-4239-991f-686f831cd348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.932372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780addb-921b-4239-991f-686f831cd348 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.932656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780addb-921b-4239-991f-686f831cd348 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.932931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780addb-921b-4239-991f-686f831cd348 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.933644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780addb-921b-4239-991f-686f831cd348 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.935008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780addb-921b-4239-991f-686f831cd348 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.007955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 579/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Tue Aug 2 10:03:03 2022] POST /identity/v3/domains => generated 267 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:06.012950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aebb28-2c9a-417f-b422-aa39c62b1491 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.013497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aebb28-2c9a-417f-b422-aa39c62b1491 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.013497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aebb28-2c9a-417f-b422-aa39c62b1491 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.014891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 591/1170] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:06.020136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f44e3e-e94a-48ed-b67e-7ebf0323f058 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.020519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f44e3e-e94a-48ed-b67e-7ebf0323f058 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.020715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f44e3e-e94a-48ed-b67e-7ebf0323f058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.038942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b4f44e3e-e94a-48ed-b67e-7ebf0323f058 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:06.040174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b4f44e3e-e94a-48ed-b67e-7ebf0323f058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:06.106869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 580/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:06.130276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aebb28-2c9a-417f-b422-aa39c62b1491 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:06.230012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aebb28-2c9a-417f-b422-aa39c62b1491 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:06.264381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b233b-6b1c-4858-a6bc-bcc942f6b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:06.264949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b233b-6b1c-4858-a6bc-bcc942f6b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.265228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b233b-6b1c-4858-a6bc-bcc942f6b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.265503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b233b-6b1c-4858-a6bc-bcc942f6b4c9 None admin] PATH_INFO: `/v3/domains/3385b6c4beaa4cf2af50a8c1130935a3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.275694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b233b-6b1c-4858-a6bc-bcc942f6b4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3385b6c4beaa4cf2af50a8c1130935a3)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:06.277164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b233b-6b1c-4858-a6bc-bcc942f6b4c9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:06.278773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 592/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Tue Aug 2 10:03:06 2022] GET /identity/v3/domains/3385b6c4beaa4cf2af50a8c1130935a3 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:06.285335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f44e3e-e94a-48ed-b67e-7ebf0323f058 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:06.353511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:06.382401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f44e3e-e94a-48ed-b67e-7ebf0323f058 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:06.416756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6d801-87ed-4172-9572-ecb29ef23050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:06.417432 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6d801-87ed-4172-9572-ecb29ef23050 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:06.417720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6d801-87ed-4172-9572-ecb29ef23050 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:06.417998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6d801-87ed-4172-9572-ecb29ef23050 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:06.418682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6d801-87ed-4172-9572-ecb29ef23050 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:06.420166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6d801-87ed-4172-9572-ecb29ef23050 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:06.518333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 581/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:03:06 2022] POST /identity/v3/projects => generated 361 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:08.536574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d866c-bb67-462f-b947-75e3eff2340a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:08.536993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d866c-bb67-462f-b947-75e3eff2340a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:08.537140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d866c-bb67-462f-b947-75e3eff2340a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:08.538504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 593/1174] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:08.543692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc231d82-c9ec-418f-b749-cd6960d8bb01 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:08.544075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc231d82-c9ec-418f-b749-cd6960d8bb01 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:08.544257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc231d82-c9ec-418f-b749-cd6960d8bb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:08.563376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fc231d82-c9ec-418f-b749-cd6960d8bb01 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:08.564565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fc231d82-c9ec-418f-b749-cd6960d8bb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:08.632063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 582/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:08.656316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d866c-bb67-462f-b947-75e3eff2340a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:08.714028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:08.758394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d866c-bb67-462f-b947-75e3eff2340a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:08.790648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde7ba1-a2ac-4781-a347-851e384e8b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:08.791244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde7ba1-a2ac-4781-a347-851e384e8b1b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:08.791531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde7ba1-a2ac-4781-a347-851e384e8b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:08.791813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde7ba1-a2ac-4781-a347-851e384e8b1b None admin] PATH_INFO: `/v3/domains/3385b6c4beaa4cf2af50a8c1130935a3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:08.792963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde7ba1-a2ac-4781-a347-851e384e8b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=3385b6c4beaa4cf2af50a8c1130935a3)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:08.794386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde7ba1-a2ac-4781-a347-851e384e8b1b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:08.796606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 594/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Tue Aug 2 10:03:08 2022] GET /identity/v3/domains/3385b6c4beaa4cf2af50a8c1130935a3 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:08.802701 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc231d82-c9ec-418f-b749-cd6960d8bb01 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:08.856345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:08.899784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc231d82-c9ec-418f-b749-cd6960d8bb01 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:08.935033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8c202-235e-4d32-af22-b7e6e116a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:08.935681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8c202-235e-4d32-af22-b7e6e116a75b None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:08.935991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8c202-235e-4d32-af22-b7e6e116a75b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:08.936266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8c202-235e-4d32-af22-b7e6e116a75b None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:08.936944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8c202-235e-4d32-af22-b7e6e116a75b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:08.938392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8c202-235e-4d32-af22-b7e6e116a75b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:09.025939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 583/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:03:08 2022] POST /identity/v3/users => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:11.053224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a6fd4-a306-4272-a413-7df60655240e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.053768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a6fd4-a306-4272-a413-7df60655240e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.053860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a6fd4-a306-4272-a413-7df60655240e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.055154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 595/1178] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:11.060241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1719f6-b8b7-4ab9-9560-c26c410a5e74 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.060615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1719f6-b8b7-4ab9-9560-c26c410a5e74 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.060806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1719f6-b8b7-4ab9-9560-c26c410a5e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.078655 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fa1719f6-b8b7-4ab9-9560-c26c410a5e74 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:11.079868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fa1719f6-b8b7-4ab9-9560-c26c410a5e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:11.146477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 584/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:11.169477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a6fd4-a306-4272-a413-7df60655240e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.270038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a6fd4-a306-4272-a413-7df60655240e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.292300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.303040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.303761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.303991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.304262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.314671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:11.319536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:11.320626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:03:11.321931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 596/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:11.326891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1719f6-b8b7-4ab9-9560-c26c410a5e74 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.368405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.425320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1719f6-b8b7-4ab9-9560-c26c410a5e74 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.458833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.459481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.459767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.460045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.460766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:11.461075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.462461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.471007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 585/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:11.476957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.576326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405c3ee-5331-4448-a51c-01235a4a4c8b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.608479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.609703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.609986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.610265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] PATH_INFO: `/v3/users/1562ba41fa914ab5885b8cb102702b01` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.623664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] RBAC: Authorizing `identity:get_user(user_id=1562ba41fa914ab5885b8cb102702b01)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.625773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.627798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 597/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/users/1562ba41fa914ab5885b8cb102702b01 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:11.633586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:11.730712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1067cc-7191-41c8-a142-684452f34ace None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:11.732259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:11.767975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:11.768595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:11.768881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:11.769159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:11.886110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:11.887544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:11.990498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 586/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:11.996383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:12.038029 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:12.092042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5039cb-530c-4acd-9ade-d4d4aebdaa0f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:12.126748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b28d2-a69a-4aea-ba00-0a3bca3e7c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:12.127693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b28d2-a69a-4aea-ba00-0a3bca3e7c60 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.127977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b28d2-a69a-4aea-ba00-0a3bca3e7c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.128253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b28d2-a69a-4aea-ba00-0a3bca3e7c60 None admin] PATH_INFO: `/v3/projects/e2d13e66d94b4b9f8f45c5941636325a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.129412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b28d2-a69a-4aea-ba00-0a3bca3e7c60 None admin] RBAC: Authorizing `identity:get_project(project_id=e2d13e66d94b4b9f8f45c5941636325a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:12.130918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b28d2-a69a-4aea-ba00-0a3bca3e7c60 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:12.132627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 598/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:11 2022] GET /identity/v3/projects/e2d13e66d94b4b9f8f45c5941636325a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:12.139924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:12.142764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:12.243377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83acf2-3f33-4272-9dfd-99264adaf7b1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:12.276957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa887c6a-39e5-4da2-8858-eabbc5d4fb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:12.277573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa887c6a-39e5-4da2-8858-eabbc5d4fb3b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:12.277859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa887c6a-39e5-4da2-8858-eabbc5d4fb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:12.278139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa887c6a-39e5-4da2-8858-eabbc5d4fb3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:12.278921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa887c6a-39e5-4da2-8858-eabbc5d4fb3b None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=e2d13e66d94b4b9f8f45c5941636325a, user.id=1562ba41fa914ab5885b8cb102702b01) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:12.279268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa887c6a-39e5-4da2-8858-eabbc5d4fb3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:12.280777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa887c6a-39e5-4da2-8858-eabbc5d4fb3b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:12.286132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 587/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:12 2022] GET /identity/v3/role_assignments?user.id=1562ba41fa914ab5885b8cb102702b01&scope.project.id=e2d13e66d94b4b9f8f45c5941636325a&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:14.288263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e4f9c-ad4c-4a03-acf2-cf9071bee634 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.288820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e4f9c-ad4c-4a03-acf2-cf9071bee634 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.288820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e4f9c-ad4c-4a03-acf2-cf9071bee634 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.290221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 599/1186] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:14.296681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86868c9-6bae-41ad-9633-e5d79189ddaa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.297075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86868c9-6bae-41ad-9633-e5d79189ddaa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.297372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86868c9-6bae-41ad-9633-e5d79189ddaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.316368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b86868c9-6bae-41ad-9633-e5d79189ddaa None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:14.317572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b86868c9-6bae-41ad-9633-e5d79189ddaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:14.380987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:14.388913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 588/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:14.411958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e4f9c-ad4c-4a03-acf2-cf9071bee634 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.509748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e4f9c-ad4c-4a03-acf2-cf9071bee634 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:14.520791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:14.543814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:14.544392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.544674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.544949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.555462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:14.560864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:14.562247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:03:14.564094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 600/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:03:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:14.569019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86868c9-6bae-41ad-9633-e5d79189ddaa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.678909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86868c9-6bae-41ad-9633-e5d79189ddaa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:14.718908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:14.719567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.719856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.720137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.720860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:14.721168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:14.722552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:14.730676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 589/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:03:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:14.737028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.838985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcb405-c781-44db-a932-63ae2ee7719e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:14.871920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:14.872548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:14.872831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:14.873103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] PATH_INFO: `/v3/users/1562ba41fa914ab5885b8cb102702b01` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:14.874293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] RBAC: Authorizing `identity:get_user(user_id=1562ba41fa914ab5885b8cb102702b01)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:14.875777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:14.877619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 601/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:14 2022] GET /identity/v3/users/1562ba41fa914ab5885b8cb102702b01 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:14.883387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:14.941210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:14.981287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a33b19-fe47-4e3c-af87-f55ec9788a5b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:15.014643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff216f-98e3-4696-909d-0e71d4e08b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:15.015311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff216f-98e3-4696-909d-0e71d4e08b7f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.015599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff216f-98e3-4696-909d-0e71d4e08b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.015878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff216f-98e3-4696-909d-0e71d4e08b7f None admin] PATH_INFO: `/v3/projects/e2d13e66d94b4b9f8f45c5941636325a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.017049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff216f-98e3-4696-909d-0e71d4e08b7f None admin] RBAC: Authorizing `identity:get_project(project_id=e2d13e66d94b4b9f8f45c5941636325a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:15.018486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff216f-98e3-4696-909d-0e71d4e08b7f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:15.020187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 590/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:14 2022] GET /identity/v3/projects/e2d13e66d94b4b9f8f45c5941636325a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:15.026182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:15.069623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:15.125077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85d2cc-b986-477d-9cc2-9357c15655f5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:15.158045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566fb0-2563-4cd3-ada7-dd116018409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:15.158705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566fb0-2563-4cd3-ada7-dd116018409e None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.158986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566fb0-2563-4cd3-ada7-dd116018409e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.159290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566fb0-2563-4cd3-ada7-dd116018409e None admin] PATH_INFO: `/v3/projects/e2d13e66d94b4b9f8f45c5941636325a/users/1562ba41fa914ab5885b8cb102702b01/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.161324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89566fb0-2563-4cd3-ada7-dd116018409e None admin] RBAC: Authorizing `identity:create_grant(project_id=e2d13e66d94b4b9f8f45c5941636325a, user_id=1562ba41fa914ab5885b8cb102702b01, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:15.162741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89566fb0-2563-4cd3-ada7-dd116018409e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:15.230049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 602/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:03:15 2022] PUT /identity/v3/projects/e2d13e66d94b4b9f8f45c5941636325a/users/1562ba41fa914ab5885b8cb102702b01/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:17.217640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98878e1-fa37-4cbb-acde-806e4bd5d474 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.218215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98878e1-fa37-4cbb-acde-806e4bd5d474 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.218215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98878e1-fa37-4cbb-acde-806e4bd5d474 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.220170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 591/1193] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:03:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:17.225422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589985c-81a2-4498-97be-16efbd4f4b57 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.225826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589985c-81a2-4498-97be-16efbd4f4b57 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.225983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589985c-81a2-4498-97be-16efbd4f4b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.244503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9589985c-81a2-4498-97be-16efbd4f4b57 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:17.245681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9589985c-81a2-4498-97be-16efbd4f4b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.311150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 603/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:03:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:03:17.334156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98878e1-fa37-4cbb-acde-806e4bd5d474 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.434444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98878e1-fa37-4cbb-acde-806e4bd5d474 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.459023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.467329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.472976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.472976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.472976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.479614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:03:17.484477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:03:17.485691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:03:17.486834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 592/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:03:17.491719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589985c-81a2-4498-97be-16efbd4f4b57 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.519200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.593704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589985c-81a2-4498-97be-16efbd4f4b57 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.627106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.628017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.628301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.628582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.629304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:17.629609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.630979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.639089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 604/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:17.645405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.745672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8e2de-8def-460d-ab03-fa429acc7f89 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.779402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.780036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.780322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.780597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] PATH_INFO: `/v3/users/1562ba41fa914ab5885b8cb102702b01` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.781749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] RBAC: Authorizing `identity:get_user(user_id=1562ba41fa914ab5885b8cb102702b01)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.783239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.784875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 593/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/users/1562ba41fa914ab5885b8cb102702b01 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:17.790770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.881877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.891319 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a0168-2e35-46e0-9271-0fa0d383905b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.925634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.926254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.926538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.926861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.991649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.993069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.079072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.089770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 605/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:03:18.095812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.150883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.196600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7f9c9-4e53-40f7-be16-8fd14607ee57 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.228926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd0e4e-ff57-498e-b26c-fa2a6d2a3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.229566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd0e4e-ff57-498e-b26c-fa2a6d2a3992 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.229853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd0e4e-ff57-498e-b26c-fa2a6d2a3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.230130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd0e4e-ff57-498e-b26c-fa2a6d2a3992 None admin] PATH_INFO: `/v3/projects/e2d13e66d94b4b9f8f45c5941636325a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.231309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd0e4e-ff57-498e-b26c-fa2a6d2a3992 None admin] RBAC: Authorizing `identity:get_project(project_id=e2d13e66d94b4b9f8f45c5941636325a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.232743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd0e4e-ff57-498e-b26c-fa2a6d2a3992 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.234403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 594/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/projects/e2d13e66d94b4b9f8f45c5941636325a => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:18.241098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.337096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7319c-032f-454f-8c80-218c92fdb1e2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.370697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677f330-2012-45e1-aa5a-823c33661b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.371334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677f330-2012-45e1-aa5a-823c33661b98 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.371621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677f330-2012-45e1-aa5a-823c33661b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.371900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677f330-2012-45e1-aa5a-823c33661b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.372743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677f330-2012-45e1-aa5a-823c33661b98 None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=e2d13e66d94b4b9f8f45c5941636325a, user.id=1562ba41fa914ab5885b8cb102702b01) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:18.373056 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677f330-2012-45e1-aa5a-823c33661b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.374552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677f330-2012-45e1-aa5a-823c33661b98 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.381104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 606/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/role_assignments?user.id=1562ba41fa914ab5885b8cb102702b01&scope.project.id=e2d13e66d94b4b9f8f45c5941636325a&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:28.224744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=39 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:28.378451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=73 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:34.472816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fcbe8-f69d-4d3d-bb1d-ea6bf210d77f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.473407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fcbe8-f69d-4d3d-bb1d-ea6bf210d77f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.473407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fcbe8-f69d-4d3d-bb1d-ea6bf210d77f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.474911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 595/1201] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:34.480289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52d86-325b-458c-8d39-658c3f3eb4dc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.480732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52d86-325b-458c-8d39-658c3f3eb4dc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.480840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52d86-325b-458c-8d39-658c3f3eb4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.499703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-52c52d86-325b-458c-8d39-658c3f3eb4dc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:34.500935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-52c52d86-325b-458c-8d39-658c3f3eb4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:34.568438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 607/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:34.593118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fcbe8-f69d-4d3d-bb1d-ea6bf210d77f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:34.695035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fcbe8-f69d-4d3d-bb1d-ea6bf210d77f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:34.728899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:34.729441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.729723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.729997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.740944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:34.742399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:34.757310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:34.758637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:34.759679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:34.759679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:34.759679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:34.759679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:34.759679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 596/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:34.764978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c52d86-325b-458c-8d39-658c3f3eb4dc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:34.862635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c52d86-325b-458c-8d39-658c3f3eb4dc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:34.892034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:34.897748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72119a78-23b6-4f81-a32d-1a85db03a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:34.898369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72119a78-23b6-4f81-a32d-1a85db03a6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:34.898656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72119a78-23b6-4f81-a32d-1a85db03a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:34.898934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72119a78-23b6-4f81-a32d-1a85db03a6e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:34.899678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72119a78-23b6-4f81-a32d-1a85db03a6e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:34.899988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72119a78-23b6-4f81-a32d-1a85db03a6e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:34.901563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72119a78-23b6-4f81-a32d-1a85db03a6e4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:34.913699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 608/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:34.919760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:34.974119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:35.022155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c68c4-4bf8-445d-b823-a6cad3c1e24a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:35.056264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514b477-0bca-4df6-b970-5527b9cfd122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:35.056890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514b477-0bca-4df6-b970-5527b9cfd122 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:35.057188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514b477-0bca-4df6-b970-5527b9cfd122 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:35.057457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514b477-0bca-4df6-b970-5527b9cfd122 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:35.058136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514b477-0bca-4df6-b970-5527b9cfd122 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:35.059610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514b477-0bca-4df6-b970-5527b9cfd122 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:35.197422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 597/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Tue Aug 2 10:04:34 2022] POST /identity/v3/users => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:04:37.253583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5c8c5-181e-4f60-b818-2ceb3bf6a8e7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:37.254135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5c8c5-181e-4f60-b818-2ceb3bf6a8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:37.254286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5c8c5-181e-4f60-b818-2ceb3bf6a8e7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:37.255735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 609/1206] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:37.261020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60935-5814-4dd3-a2f2-d392b1125733 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:37.261441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60935-5814-4dd3-a2f2-d392b1125733 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:37.261587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60935-5814-4dd3-a2f2-d392b1125733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:37.280573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b8d60935-5814-4dd3-a2f2-d392b1125733 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:37.282319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b8d60935-5814-4dd3-a2f2-d392b1125733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:37.350464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 598/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:37.376063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5c8c5-181e-4f60-b818-2ceb3bf6a8e7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:37.483920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5c8c5-181e-4f60-b818-2ceb3bf6a8e7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:37.518460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:37.519036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:37.519352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:37.519628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:37.530429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:37.535902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:37.537057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:37.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 610/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:37.543085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60935-5814-4dd3-a2f2-d392b1125733 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:37.581997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:37.645116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60935-5814-4dd3-a2f2-d392b1125733 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:37.677485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:37.678098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:37.678379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:37.678659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:37.679394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:37.679699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:37.681061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:37.689279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 599/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:37.695024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:37.745791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:37.795649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a8ead-b34b-4c94-a30c-c44c441a79a2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:37.828695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:37.829343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:37.829635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:37.829915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:37.840137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:37.841590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:37.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:37.856386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:37.857336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:37.857336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:37.857336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:37.857336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:37.857336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 611/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:37.862741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:37.961349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:37.964396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e04424-419a-4464-aa7b-8467ab98e59d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:37.997810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:37.998426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:37.998706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:37.998984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:37.999722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:38.000030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:38.001417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:38.013087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 600/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:38.019237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:38.118512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949847a4-b82d-4e6d-9a3f-ab0934b3a377 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:38.120000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:38.152388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:38.152978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:38.153263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:38.153542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:38.161686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:38.163115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:04:38.174624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:38.181967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:38.183155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 612/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:38.183947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:38.282094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224c5fc-abf3-41c3-afa4-f3ba38130280 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:38.316798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:38.317426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:38.317715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:38.317995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:38.318867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:38.319180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:38.320696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:38.337677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 601/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Tue Aug 2 10:04:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:38.344079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:38.442191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f2e1a-6460-4ed3-8a9b-f6d28a771f75 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:38.474533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:38.475162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:38.475489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:38.475771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:38.504746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:38.541979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:38.543463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:38.642759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 613/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:38.648563 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:38.667325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:38.746027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f77e9c-619d-4e6d-80ab-d8af432b2a89 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:38.780636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:38.781334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:38.781620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:38.781910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:38.791887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:38.793424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:38.808544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:38.814804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:38.815930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:38.815930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:38.815930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:38.815930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:38.815930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 602/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:38.816216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:38.876391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:38.915729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e559904-91c8-4f49-8d2a-956ec0909171 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:38.950896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:38.951546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:38.951834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:38.952113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:38.952804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:38.953117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:38.954490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:38.965432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 614/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:38.973933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:39.051716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:39.082180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ea9f-6641-4d2e-9a57-eaf461b83d8b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:39.115948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:39.116622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:39.116903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:39.117176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:39.123588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:39.125021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:04:39.136170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:39.137233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:39.138400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 603/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:04:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:39.143474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:39.243948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c6233-7fce-4c19-a223-617e2001e90a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:39.266294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:39.277503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1eeff8-be6e-42e4-8a49-c80363449d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:39.278141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eeff8-be6e-42e4-8a49-c80363449d01 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:39.278421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eeff8-be6e-42e4-8a49-c80363449d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:39.278697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eeff8-be6e-42e4-8a49-c80363449d01 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:39.279469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1eeff8-be6e-42e4-8a49-c80363449d01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:39.279781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1eeff8-be6e-42e4-8a49-c80363449d01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:39.281741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1eeff8-be6e-42e4-8a49-c80363449d01 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:39.293835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 615/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:39.301489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:39.409188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd704f-401f-45fc-a7d1-1b986a543b1a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:39.437349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:39.443306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84654dd5-d17d-4dc2-9614-200c54d3d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:39.443933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654dd5-d17d-4dc2-9614-200c54d3d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:39.444219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654dd5-d17d-4dc2-9614-200c54d3d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:39.444499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654dd5-d17d-4dc2-9614-200c54d3d3b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:39.445293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654dd5-d17d-4dc2-9614-200c54d3d3b9 None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=e68c847a5a60461a840679bbefc8ab6d) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:39.445602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654dd5-d17d-4dc2-9614-200c54d3d3b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:39.447110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654dd5-d17d-4dc2-9614-200c54d3d3b9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:39.452600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 604/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:04:39 2022] GET /identity/v3/role_assignments?user.id=e68c847a5a60461a840679bbefc8ab6d&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:41.462408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8411a-5db1-4971-80f9-deb45d0c6801 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:41.463287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8411a-5db1-4971-80f9-deb45d0c6801 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:41.463490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8411a-5db1-4971-80f9-deb45d0c6801 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:41.464915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 616/1220] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:41.470737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7097a46-d268-4f93-b88a-af84488a52b2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:41.471270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7097a46-d268-4f93-b88a-af84488a52b2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:41.471414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7097a46-d268-4f93-b88a-af84488a52b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:41.489822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b7097a46-d268-4f93-b88a-af84488a52b2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:41.491069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b7097a46-d268-4f93-b88a-af84488a52b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:41.558972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 605/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:41.582221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8411a-5db1-4971-80f9-deb45d0c6801 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:41.686555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8411a-5db1-4971-80f9-deb45d0c6801 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:41.720219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:41.720796 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:41.721079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:41.721356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:41.731402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:41.736828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:41.737932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:41.738991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 617/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:04:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:41.747153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7097a46-d268-4f93-b88a-af84488a52b2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:41.842142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7097a46-d268-4f93-b88a-af84488a52b2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:41.865740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:41.877403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:41.878033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:41.878323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:41.878618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:41.879370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:41.879680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:41.881092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:41.889175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 606/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:04:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:41.894973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:41.938568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:41.992531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6aa7-010d-4f84-8ab5-5863b6cc5312 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:42.024885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:42.025513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:42.025794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:42.026069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:42.036287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:42.037776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:42.052674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:42.053576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:42.054455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:42.054455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:42.054455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:42.054455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:42.054455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 618/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:42.059712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:42.165265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f9c20-ea98-4ada-8b5c-cbfb11f49f7c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:42.199480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:42.200094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:42.200383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:42.200665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:42.201366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:42.201676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:42.203674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:42.215607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 607/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:42.221631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:42.316937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:42.323606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdac61a-55d9-4c2b-ab3a-3b66a53ec5ce None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:42.356616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:42.357253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:42.357541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:42.357820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:42.365820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:42.367322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:04:42.379074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:42.380186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:42.381288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 619/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:42.386093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:42.434350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:42.486483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c04bd-129f-4e39-ac38-14629ab9ed8f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:42.519886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:42.520504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:42.520785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:42.521058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:42.521850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:42.522161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:42.523675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:42.540971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 608/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Tue Aug 2 10:04:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:42.546761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:42.646644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8538c-d9fb-4483-a7c2-20f198dac3b3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:42.680241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:42.680884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:42.681171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:42.681451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:42.692336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:42.693808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:42.707952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:42.708999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:42.714871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:42.714871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:42.714871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:42.714871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:42.714871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 620/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:42.715252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:42.814697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d99215-aa14-4878-a691-8763a6ccb8ae None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:42.817073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:42.848104 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:42.848734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:42.849021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:42.849301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:42.849991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:42.850302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:42.851724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:42.863378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 609/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:42.870776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:42.881013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:42.970521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b21b9-25a1-425c-90bc-4e6f0c2f0513 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:43.004331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:43.004970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:43.005251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:43.005522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:43.012114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:43.013832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:04:43.025170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:43.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:43.027388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 621/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:43.032788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:43.133798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d1b7-4f14-4b57-8f02-423d9c5e364b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:43.168407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6c5b3-9089-4b51-a2c3-b1e732c7eeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:43.169028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c5b3-9089-4b51-a2c3-b1e732c7eeda None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:43.169315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c5b3-9089-4b51-a2c3-b1e732c7eeda None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:43.169593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c5b3-9089-4b51-a2c3-b1e732c7eeda None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:43.170397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6c5b3-9089-4b51-a2c3-b1e732c7eeda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:43.170702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6c5b3-9089-4b51-a2c3-b1e732c7eeda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:43.172662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6c5b3-9089-4b51-a2c3-b1e732c7eeda None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:43.184516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 610/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:43.191108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:43.257550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:43.292590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1260e-4c79-4c9f-b7fc-8c234d64875b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:43.324680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508a96f-9b56-444c-8caa-2bfdf0ccf659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:43.325340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508a96f-9b56-444c-8caa-2bfdf0ccf659 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:43.325625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508a96f-9b56-444c-8caa-2bfdf0ccf659 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:43.325903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508a96f-9b56-444c-8caa-2bfdf0ccf659 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/e68c847a5a60461a840679bbefc8ab6d/roles/b0563004ff664008b6db05941cf9c9b9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:43.340242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508a96f-9b56-444c-8caa-2bfdf0ccf659 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=e68c847a5a60461a840679bbefc8ab6d, role_id=b0563004ff664008b6db05941cf9c9b9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:43.341706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508a96f-9b56-444c-8caa-2bfdf0ccf659 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:43.486461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 622/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:04:43 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/e68c847a5a60461a840679bbefc8ab6d/roles/b0563004ff664008b6db05941cf9c9b9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:45.484295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887ca4-d58a-4604-942e-829f3e275f1c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:45.484900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887ca4-d58a-4604-942e-829f3e275f1c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:45.484900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887ca4-d58a-4604-942e-829f3e275f1c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:45.486193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 611/1233] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:45.491484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed584a5-a7c0-4d4c-a016-136f6554acf1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:45.493243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed584a5-a7c0-4d4c-a016-136f6554acf1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:45.493243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed584a5-a7c0-4d4c-a016-136f6554acf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:45.510112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bed584a5-a7c0-4d4c-a016-136f6554acf1 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:45.511372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bed584a5-a7c0-4d4c-a016-136f6554acf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:45.578247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 623/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:04:45.602760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2887ca4-d58a-4604-942e-829f3e275f1c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:45.629335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:45.706175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2887ca4-d58a-4604-942e-829f3e275f1c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:45.751692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:45.752378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:45.752666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:45.752949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:45.764158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:45.769265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:45.770461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:45.771773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 612/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Tue Aug 2 10:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:45.776583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed584a5-a7c0-4d4c-a016-136f6554acf1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:45.823952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:45.876193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed584a5-a7c0-4d4c-a016-136f6554acf1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:45.909619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:45.910458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:45.910767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:45.911013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:45.911758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:45.912063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:45.913424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:45.921155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 624/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Tue Aug 2 10:04:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:45.927042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:46.010253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:46.026847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7009d44-8a22-474f-9fef-96c872902df8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:46.061058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:46.061772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:46.062071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:46.062349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:46.072596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:46.074063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:46.089006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:46.095631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:46.096729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:46.096729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:46.096729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:46.096729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:46.096729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 613/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:46.097082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:46.197592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88a013-1eee-4c93-b0f4-4ba922166f17 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:46.206474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:46.230163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:46.230782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:46.232473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:46.232473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:46.232473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:46.232473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:46.233783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:46.246401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 625/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:46.252841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:46.352419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91625e48-6d50-4889-b38d-4df5f2b161f1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:46.387610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:46.388256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:46.388537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:46.389065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:46.391804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:46.396759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:46.398273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:04:46.409240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:46.412440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:46.413478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 614/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:46.416741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:46.517410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866457-dffb-46c5-8712-90643e72a62b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:46.552654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:46.553275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:46.556962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:46.556962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:46.556962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:46.556962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:46.556962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:46.572921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 626/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Tue Aug 2 10:04:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:04:46.578748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:46.678586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bbece-eb16-4606-a5f2-8efe73ca9e43 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:46.711312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:46.711862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:46.712146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:46.712421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:46.772717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:46.779101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:46.780582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:46.875522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 615/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:46.881531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:46.910449 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:46.980322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2eacf-af89-4541-9c1d-df9741128558 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:47.014036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:47.014669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:47.014953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:47.015255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:47.025811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:47.027274 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:47.042334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:47.045734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:47.046853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:47.046853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:47.046853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:47.046853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:47.046853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 627/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:47.049425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:47.142723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:47.148521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f92e2a-3b18-4a09-a928-0f9c0928cd3e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:47.181728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:47.182350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:47.182631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:47.182904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:47.183609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:47.183912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:47.185266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:47.196908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 616/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:47.202628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:47.283337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:47.310619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65a959-7bcd-40e8-a377-64db6fa59ec7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:47.348404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:47.348924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:47.349211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:47.349492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:47.356226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:47.357708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:04:47.368944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:47.382762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:47.383912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 628/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:47.384532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:47.477297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc838a-81f9-4c02-b163-383ed9e478af None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:47.512447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755d88c-c5f4-4bd9-bed3-40f9c8422902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:47.513082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755d88c-c5f4-4bd9-bed3-40f9c8422902 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:47.513371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755d88c-c5f4-4bd9-bed3-40f9c8422902 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:47.513654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755d88c-c5f4-4bd9-bed3-40f9c8422902 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:47.514667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755d88c-c5f4-4bd9-bed3-40f9c8422902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:47.514984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755d88c-c5f4-4bd9-bed3-40f9c8422902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:47.517024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755d88c-c5f4-4bd9-bed3-40f9c8422902 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:47.529437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 617/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:47.535696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:47.637916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4521e-7d1d-4301-8189-2b2cb6363efe None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:47.671990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b93bed-b5b0-42aa-8515-d37b25e8d49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:47.673148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b93bed-b5b0-42aa-8515-d37b25e8d49f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:47.673430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b93bed-b5b0-42aa-8515-d37b25e8d49f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:47.673704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b93bed-b5b0-42aa-8515-d37b25e8d49f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:47.674503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b93bed-b5b0-42aa-8515-d37b25e8d49f None admin] RBAC: Adding query filter params (role.id=b0563004ff664008b6db05941cf9c9b9, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=e68c847a5a60461a840679bbefc8ab6d) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:47.674817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b93bed-b5b0-42aa-8515-d37b25e8d49f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:47.676292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b93bed-b5b0-42aa-8515-d37b25e8d49f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:47.683226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 629/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:04:47 2022] GET /identity/v3/role_assignments?user.id=e68c847a5a60461a840679bbefc8ab6d&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=b0563004ff664008b6db05941cf9c9b9 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:49.749968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534a008-6a37-4da0-97bb-4e92e9da8fe2 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:49.750512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534a008-6a37-4da0-97bb-4e92e9da8fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:49.750512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534a008-6a37-4da0-97bb-4e92e9da8fe2 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:49.751966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 618/1247] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:49.756936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bac5e-282f-4b7e-a526-1ee26c7533cf None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:49.757364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bac5e-282f-4b7e-a526-1ee26c7533cf None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:49.757491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bac5e-282f-4b7e-a526-1ee26c7533cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:49.763793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:49.779434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f20bac5e-282f-4b7e-a526-1ee26c7533cf None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:49.780620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f20bac5e-282f-4b7e-a526-1ee26c7533cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:49.847736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 630/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:49.870474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534a008-6a37-4da0-97bb-4e92e9da8fe2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:49.888953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:49.976184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534a008-6a37-4da0-97bb-4e92e9da8fe2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:50.009249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:50.009879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:50.010159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:50.010461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:50.021218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:50.026095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:50.027332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:50.033685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 619/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:50.034583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bac5e-282f-4b7e-a526-1ee26c7533cf None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:50.132268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bac5e-282f-4b7e-a526-1ee26c7533cf None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:50.162000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:50.166796 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:50.167447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:50.167736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:50.168016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:50.168730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:50.169039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:50.170407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:50.178523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 631/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:50.184489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:50.263009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:50.286077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ffd4-255e-4938-b2aa-4382f5abc419 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:50.319168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:50.319854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:50.320137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:50.320410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:50.330883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:50.332358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:50.347267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:50.348347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:50.349249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:50.349249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:50.349249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:50.349249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:50.349249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 620/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:50.354615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:50.453360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43faeb-0c8f-4179-bf89-2116a1b7a168 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:50.486688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:50.487323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:50.487611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:50.487892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:50.488578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:50.488888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:50.490259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:50.502138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 632/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:04:50.509795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:50.613016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a760-1420-4499-aee1-5d13e9462fd3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:50.640512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:50.650759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:50.651453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:50.651741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:50.652020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:50.660379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:50.661812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:04:50.672966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:50.673906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:50.674879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 621/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:50.680113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:50.736054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:50.779179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a729f2-51ef-427c-b55b-00cbedf464ff None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:50.813356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:50.813977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:50.814268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:50.814550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:50.815369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:50.815685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:50.817115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:50.834234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 633/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Tue Aug 2 10:04:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:50.840091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:50.940680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fe6e6-943a-47e7-b65f-cf589ec9f2b7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:50.975035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:50.975755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:50.976040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:50.976318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:51.031349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:51.044044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:51.045504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:51.144593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 622/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:51.150589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:51.250530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0aa7-4f17-4540-b206-2f59b30dd4c8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:51.263455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:51.285196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:51.285911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:51.286193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:51.286471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:51.296192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:51.298257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:51.313770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:51.314917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:51.316094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:51.316094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:51.316094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:51.316094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:51.316094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 634/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:51.320789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:51.407565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:51.419201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069546c1-6de3-468f-96c2-f69dcf69499e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:51.452548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:51.453163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:51.453450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:51.453726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:51.454646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:51.454950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:51.456366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:51.467248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 623/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:51.474774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:51.576468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8b689-4d26-495d-b9dd-4b3931c26386 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:51.608933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:51.609573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:51.609877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:51.610778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:51.617504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:51.618970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:04:51.630714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:51.631792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:51.633061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 635/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:51.637905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:51.736454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dabf8-c06c-4499-88a4-6df795b79d5b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:51.771639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6dabf-c89c-4091-a3b4-d2274134d9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:51.772277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6dabf-c89c-4091-a3b4-d2274134d9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:51.772559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6dabf-c89c-4091-a3b4-d2274134d9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:51.772830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6dabf-c89c-4091-a3b4-d2274134d9b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:51.773569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6dabf-c89c-4091-a3b4-d2274134d9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:51.773875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6dabf-c89c-4091-a3b4-d2274134d9b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:51.775338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6dabf-c89c-4091-a3b4-d2274134d9b5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:51.787322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 624/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:51.793586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:51.819558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:51.892604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d905b35-2209-4175-b080-bd59baa49604 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:51.924595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de0280-c35c-47ea-935b-620cb0cd667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:51.925260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de0280-c35c-47ea-935b-620cb0cd667f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:51.925546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de0280-c35c-47ea-935b-620cb0cd667f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:51.925819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de0280-c35c-47ea-935b-620cb0cd667f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:51.926595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de0280-c35c-47ea-935b-620cb0cd667f None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=e68c847a5a60461a840679bbefc8ab6d) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:51.926904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de0280-c35c-47ea-935b-620cb0cd667f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:51.928381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de0280-c35c-47ea-935b-620cb0cd667f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:51.933644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 636/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:04:51 2022] GET /identity/v3/role_assignments?user.id=e68c847a5a60461a840679bbefc8ab6d&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:53.942694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658a4a9-24ff-4b9b-9790-9ae4e17e2587 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:53.943253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658a4a9-24ff-4b9b-9790-9ae4e17e2587 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:53.943253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658a4a9-24ff-4b9b-9790-9ae4e17e2587 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:53.944569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 625/1261] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:53.949563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facd880-6119-4157-a4be-a35fb659f86e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:53.950014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facd880-6119-4157-a4be-a35fb659f86e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:53.950127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facd880-6119-4157-a4be-a35fb659f86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:53.969055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0facd880-6119-4157-a4be-a35fb659f86e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:53.970332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0facd880-6119-4157-a4be-a35fb659f86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:54.037762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 637/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:54.060471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658a4a9-24ff-4b9b-9790-9ae4e17e2587 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:54.071530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:54.162838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658a4a9-24ff-4b9b-9790-9ae4e17e2587 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:54.196444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:54.197006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:54.197292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:54.197569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:54.207658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:54.212688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:54.213838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:54.214846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 626/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:54.220514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facd880-6119-4157-a4be-a35fb659f86e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:54.239710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:54.318304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facd880-6119-4157-a4be-a35fb659f86e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:54.354043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:54.354717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:54.355000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:54.355308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:54.356065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:54.356421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:54.357787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:54.366588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 638/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:54.372423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:54.439882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:54.472107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237681-3cc4-42fb-871b-657453a9b191 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:54.505508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:54.506139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:54.506420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:54.506695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:54.516869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:54.518317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:54.534038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:54.535085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:54.536225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:54.536225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:54.536225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:54.536225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:54.536225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 627/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:54.541137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:54.621467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:54.640476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390496d5-d6aa-4d2d-b05e-3a5aa188158c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:54.673598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:54.674212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:54.674495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:54.674769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:54.675479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:54.675784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:54.677137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:54.690624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 639/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:54.696110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:54.794345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f205c08-0a60-4e24-9a9c-0efe51699492 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:54.814553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:54.828589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:54.829841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:54.830122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:54.830399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:54.837817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:54.839328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:04:54.850478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:54.855111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:54.856423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 628/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:54.857700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:54.958489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2123-77aa-4870-ac08-d4c5f2f074ac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:54.994726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:54.995464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:54.995747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:54.996023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:54.996810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:54.997122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:54.998556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:55.007382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:55.016989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 640/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Tue Aug 2 10:04:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:55.022269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:55.124634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c110c26-7fa8-4f42-b9c1-c2610d00b124 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:55.158096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:55.158733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:55.159014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:55.159315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:55.169713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:55.171176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:55.185026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:55.186219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:55.187354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:55.187354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:55.187354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:55.187354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:55.191028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 629/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:55.192217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:55.291591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372800-24ff-4751-9874-00e00003d60a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:55.324354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:55.324969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:55.325254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:55.325534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:55.326303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:55.326718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:55.328275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:55.340178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 641/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:55.346608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:55.378191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:55.448176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0073323-c641-4d2e-adb2-8675535873ad None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:55.482226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:55.482873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:55.483159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:55.483468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:55.490226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:55.491758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:04:55.503654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:55.506219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:55.507206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 630/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:55.511022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:55.573298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:55.615881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631281d-5c9f-4f83-b421-750fcb955a51 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:55.650549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06928fab-7f7a-4956-8321-079811ce5b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:55.651113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06928fab-7f7a-4956-8321-079811ce5b02 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:55.651438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06928fab-7f7a-4956-8321-079811ce5b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:55.651718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06928fab-7f7a-4956-8321-079811ce5b02 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:55.652460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06928fab-7f7a-4956-8321-079811ce5b02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:55.652775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06928fab-7f7a-4956-8321-079811ce5b02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:55.654194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06928fab-7f7a-4956-8321-079811ce5b02 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:55.668932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 642/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:55.675632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:55.768088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:55.776535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612140d-5fb5-48a0-a49b-d2ddf109b907 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:55.810498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a80dee-6fbc-418a-811b-1fc6d0177bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:55.811234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80dee-6fbc-418a-811b-1fc6d0177bb1 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:55.811561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80dee-6fbc-418a-811b-1fc6d0177bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:55.811874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80dee-6fbc-418a-811b-1fc6d0177bb1 None admin] PATH_INFO: `/v3/projects/aa12cb147a104399bade5d920da1304b/users/e68c847a5a60461a840679bbefc8ab6d/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:55.814005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a80dee-6fbc-418a-811b-1fc6d0177bb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa12cb147a104399bade5d920da1304b, user_id=e68c847a5a60461a840679bbefc8ab6d, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:55.816787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a80dee-6fbc-418a-811b-1fc6d0177bb1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:55.933367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 631/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Tue Aug 2 10:04:55 2022] PUT /identity/v3/projects/aa12cb147a104399bade5d920da1304b/users/e68c847a5a60461a840679bbefc8ab6d/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:57.989795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda4e7-1d3c-41c7-9f18-282dd5a249dd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.990297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda4e7-1d3c-41c7-9f18-282dd5a249dd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.990297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda4e7-1d3c-41c7-9f18-282dd5a249dd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.991647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 643/1274] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:57.997037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0a21-dfa4-4f65-872d-dfc78fc3b583 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.997484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0a21-dfa4-4f65-872d-dfc78fc3b583 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.997600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0a21-dfa4-4f65-872d-dfc78fc3b583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:58.016973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a9cf0a21-dfa4-4f65-872d-dfc78fc3b583 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:58.018778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a9cf0a21-dfa4-4f65-872d-dfc78fc3b583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:58.085798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 632/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:04:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:04:58.109425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda4e7-1d3c-41c7-9f18-282dd5a249dd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:58.211498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda4e7-1d3c-41c7-9f18-282dd5a249dd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:58.228432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:58.245742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:58.246375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:58.246752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:58.247061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:58.257794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:04:58.262643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:58.271490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:58.272550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 644/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Tue Aug 2 10:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:58.273459 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf0a21-dfa4-4f65-872d-dfc78fc3b583 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:58.350385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:58.368111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf0a21-dfa4-4f65-872d-dfc78fc3b583 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:58.403755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:58.404371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:58.404652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:58.404933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:58.405663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:58.405967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:58.407356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:58.415154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 633/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Tue Aug 2 10:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:58.420892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:58.520377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620b25-57d5-46c4-a332-67737db764bf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:58.557123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:58.557763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:58.558045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:58.558375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:58.568179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:58.569614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:58.584597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:58.585673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:58.586829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:58.586829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:58.586829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:58.586829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:58.586829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 645/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:58.591548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:58.691204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9569ff3-eb5a-4128-bc23-904a504f7237 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:58.724237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:58.724848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:58.725131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:58.725410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:58.726087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:58.726388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:58.727775 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:58.739254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 634/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:58.745083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:58.777018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:58.853621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc2001-a5f7-49d8-8453-caed57944a1d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:58.893127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:58.893845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:58.894131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:58.894409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:58.902378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:58.903858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:04:58.914864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:58.915902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:04:58.916868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 646/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:58.922081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:58.926554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:59.025120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc0508-d3d3-4638-a823-4b0a9069dc80 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:59.059539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:59.060089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:59.060372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:59.060645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:59.061424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:59.061784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:59.063222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:59.082557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 635/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Tue Aug 2 10:04:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:59.088588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:59.179064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:59.188538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9ec91-84a7-4f69-b2d8-b69b0ceda3a4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:59.221822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:59.222440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:59.222724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:59.223055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:59.290401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:59.293281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:59.382998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:59.393414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 647/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Tue Aug 2 10:04:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:04:59.399390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:59.461285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:59.498562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5cb07-9994-44c2-95b4-57cbf6cb3b6c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:59.531088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:59.531811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:59.532031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:59.532306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:59.542866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:59.544391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:04:59.560049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:59.561217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:59.562179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:04:59.562179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:04:59.562179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:04:59.562179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:59.562179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 636/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:59.566971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:59.685968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca9c07-b40c-4f8d-9863-9d3537c841b6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:59.723347 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:59.723963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:59.724248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:59.724526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:59.725213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:59.725522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:59.726895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:59.741379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 648/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Tue Aug 2 10:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:59.747001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:59.847072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc9aee-fb42-4685-b454-19f1de8bf65c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:59.860784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:59.883696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:59.884353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:59.884634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:59.884910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:59.892050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:59.894060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:04:59.905174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:04:59.906110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:04:59.907301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 637/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Tue Aug 2 10:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:04:59.913130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:59.968607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:00.020524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b0fea-3726-41ee-9163-efa02d811890 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:00.055920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fec442-a86d-4951-b1b5-b805f25d435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:00.056600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec442-a86d-4951-b1b5-b805f25d435a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:00.056886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec442-a86d-4951-b1b5-b805f25d435a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:00.057164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec442-a86d-4951-b1b5-b805f25d435a None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:00.058113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec442-a86d-4951-b1b5-b805f25d435a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:00.058427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec442-a86d-4951-b1b5-b805f25d435a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:00.059979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec442-a86d-4951-b1b5-b805f25d435a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:00.072994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 649/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Tue Aug 2 10:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:00.079126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:00.184101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45acc-7f6a-4dbc-ac69-d3d6839aa5aa None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:00.217874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9d144-980c-4068-8b51-c43a6042c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:00.218493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d144-980c-4068-8b51-c43a6042c80b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:00.218830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d144-980c-4068-8b51-c43a6042c80b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:00.219059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d144-980c-4068-8b51-c43a6042c80b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:00.219881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9d144-980c-4068-8b51-c43a6042c80b None admin] RBAC: Adding query filter params (role.id=5d7ec7f52d40437c82607e4299a3bdd6, scope.project.id=aa12cb147a104399bade5d920da1304b, user.id=e68c847a5a60461a840679bbefc8ab6d) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:00.220197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9d144-980c-4068-8b51-c43a6042c80b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:00.222243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9d144-980c-4068-8b51-c43a6042c80b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:00.228438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 638/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Tue Aug 2 10:05:00 2022] GET /identity/v3/role_assignments?user.id=e68c847a5a60461a840679bbefc8ab6d&scope.project.id=aa12cb147a104399bade5d920da1304b&role.id=5d7ec7f52d40437c82607e4299a3bdd6 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:02.281921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8035ecd-2fdd-477a-ab9f-4cf173bd6946 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:02.282484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8035ecd-2fdd-477a-ab9f-4cf173bd6946 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:02.282484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8035ecd-2fdd-477a-ab9f-4cf173bd6946 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:02.283897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 650/1288] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:02.289141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58778e24-fbdf-4282-90d1-dd738dbf3423 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:02.289575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58778e24-fbdf-4282-90d1-dd738dbf3423 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:02.289751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58778e24-fbdf-4282-90d1-dd738dbf3423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:02.307944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-58778e24-fbdf-4282-90d1-dd738dbf3423 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:02.309123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-58778e24-fbdf-4282-90d1-dd738dbf3423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:02.339105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:02.375411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 639/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:02.398630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8035ecd-2fdd-477a-ab9f-4cf173bd6946 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:02.506467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8035ecd-2fdd-477a-ab9f-4cf173bd6946 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:02.518934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:02.544229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:02.544815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:02.545097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:02.545401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:02.546032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:02.547440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:02.556401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:02.557621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:02.558517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:02.566223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 651/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:05:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:02.566368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58778e24-fbdf-4282-90d1-dd738dbf3423 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:02.662205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58778e24-fbdf-4282-90d1-dd738dbf3423 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:02.696419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed674f-377d-432f-bb0e-8bacee8df2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:02.697039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed674f-377d-432f-bb0e-8bacee8df2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:02.697325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed674f-377d-432f-bb0e-8bacee8df2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:02.697603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed674f-377d-432f-bb0e-8bacee8df2f2 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:02.698275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed674f-377d-432f-bb0e-8bacee8df2f2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:02.698583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed674f-377d-432f-bb0e-8bacee8df2f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:02.699974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed674f-377d-432f-bb0e-8bacee8df2f2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:02.704722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 640/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Tue Aug 2 10:05:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:02.710632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:02.811366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2035b-0c1e-409d-a889-2fe99a9fd186 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:02.844462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbe885-6a02-42af-bf56-fb6f5a97e99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:02.845073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe885-6a02-42af-bf56-fb6f5a97e99a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:02.845755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe885-6a02-42af-bf56-fb6f5a97e99a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:02.845755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe885-6a02-42af-bf56-fb6f5a97e99a None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:02.846380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe885-6a02-42af-bf56-fb6f5a97e99a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:02.846687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe885-6a02-42af-bf56-fb6f5a97e99a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:02.848095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe885-6a02-42af-bf56-fb6f5a97e99a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:02.852501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 652/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Tue Aug 2 10:05:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:04.842735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3fbf-4520-4fbd-a73f-4eb0cfbb64dd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:04.843295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3fbf-4520-4fbd-a73f-4eb0cfbb64dd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:04.843295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3fbf-4520-4fbd-a73f-4eb0cfbb64dd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:04.844589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 641/1293] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:04.850230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0babe61-b23b-4451-8d1c-b1cddbc58adb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:04.850647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0babe61-b23b-4451-8d1c-b1cddbc58adb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:04.850901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0babe61-b23b-4451-8d1c-b1cddbc58adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:04.869909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f0babe61-b23b-4451-8d1c-b1cddbc58adb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:04.871246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f0babe61-b23b-4451-8d1c-b1cddbc58adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:04.900814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:04.937021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 653/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:04.960259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e3fbf-4520-4fbd-a73f-4eb0cfbb64dd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:04.992634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:05.066795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e3fbf-4520-4fbd-a73f-4eb0cfbb64dd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:05.102777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40052a6d-054d-45f2-81ed-ec26cb86f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:05.102777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052a6d-054d-45f2-81ed-ec26cb86f5ec None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:05.102777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052a6d-054d-45f2-81ed-ec26cb86f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:05.103224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052a6d-054d-45f2-81ed-ec26cb86f5ec None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:05.104112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40052a6d-054d-45f2-81ed-ec26cb86f5ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:05.105143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40052a6d-054d-45f2-81ed-ec26cb86f5ec None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:05.212203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 642/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Tue Aug 2 10:05:04 2022] POST /identity/v3/services => generated 246 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:05:07.257859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291c915-7677-4f52-9317-5a7488f8faf4 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:07.258430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291c915-7677-4f52-9317-5a7488f8faf4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:07.258430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291c915-7677-4f52-9317-5a7488f8faf4 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:07.259914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 654/1296] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:07.265548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002b94-83ef-43f2-9653-685cadc6689a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:07.270494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002b94-83ef-43f2-9653-685cadc6689a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:07.270494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002b94-83ef-43f2-9653-685cadc6689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:07.285068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-48002b94-83ef-43f2-9653-685cadc6689a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:07.286250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-48002b94-83ef-43f2-9653-685cadc6689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:07.357894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 643/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:07.381248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291c915-7677-4f52-9317-5a7488f8faf4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:07.482264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291c915-7677-4f52-9317-5a7488f8faf4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:07.522118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:07.522715 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:07.523000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:07.523305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] PATH_INFO: `/v3/services/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:07.523935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:07.525410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:07.534528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:07.535731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:07.536735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 655/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:05:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:07.542212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48002b94-83ef-43f2-9653-685cadc6689a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:07.560997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:07.647901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48002b94-83ef-43f2-9653-685cadc6689a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:07.683068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:07.683829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:07.684147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:07.684494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:07.685239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:07.685572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:07.687121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:07.693335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 644/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Tue Aug 2 10:05:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:07.699640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:07.736964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:07.813030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea32e10-8077-4c4e-bd0a-86cc442fa9ac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:07.848072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ac917-cf75-437d-b659-46546936f730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:07.848696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ac917-cf75-437d-b659-46546936f730 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:07.848981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ac917-cf75-437d-b659-46546936f730 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:07.849260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ac917-cf75-437d-b659-46546936f730 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:07.850016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ac917-cf75-437d-b659-46546936f730 None admin] RBAC: Adding query filter params (interface=public, service_id=a162b245686a45aa95d97a9680af5232, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:07.850330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ac917-cf75-437d-b659-46546936f730 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:07.851740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ac917-cf75-437d-b659-46546936f730 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:07.857250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 656/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Tue Aug 2 10:05:07 2022] GET /identity/v3/endpoints?service_id=a162b245686a45aa95d97a9680af5232&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:07.863754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:07.955600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:07.976789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fef3fd-ed75-4e3b-bf7d-876f4d882223 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:08.014429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5f2c1-d187-4550-88bc-9d3ce3b630c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:08.015080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f2c1-d187-4550-88bc-9d3ce3b630c6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:08.015374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f2c1-d187-4550-88bc-9d3ce3b630c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:08.015652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f2c1-d187-4550-88bc-9d3ce3b630c6 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:08.016341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5f2c1-d187-4550-88bc-9d3ce3b630c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:08.017996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5f2c1-d187-4550-88bc-9d3ce3b630c6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:08.024853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 645/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Tue Aug 2 10:05:07 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:05:10.029120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388cdf5-0954-4dbd-ac7d-f140bf49e68a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:10.029653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388cdf5-0954-4dbd-ac7d-f140bf49e68a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:10.029653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388cdf5-0954-4dbd-ac7d-f140bf49e68a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:10.031041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 657/1302] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:05:10.036214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196e3fd-c47c-448a-a2ad-ec91e96a6667 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:10.036621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196e3fd-c47c-448a-a2ad-ec91e96a6667 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:10.036802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196e3fd-c47c-448a-a2ad-ec91e96a6667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:10.054874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5196e3fd-c47c-448a-a2ad-ec91e96a6667 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:10.056089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5196e3fd-c47c-448a-a2ad-ec91e96a6667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:10.121371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 646/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:05:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:05:10.144471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388cdf5-0954-4dbd-ac7d-f140bf49e68a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:10.238378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:10.246007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388cdf5-0954-4dbd-ac7d-f140bf49e68a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:10.285107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:10.285752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:10.286073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:10.286360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:10.287100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:10.288830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:05:10.298891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:05:10.300411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:05:10.304624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 658/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Tue Aug 2 10:05:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:05:10.307084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196e3fd-c47c-448a-a2ad-ec91e96a6667 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:10.387695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:10.409959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196e3fd-c47c-448a-a2ad-ec91e96a6667 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:10.445040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e37c46-0de1-494f-9e7a-13ab350607a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:10.445740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37c46-0de1-494f-9e7a-13ab350607a9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:10.446021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37c46-0de1-494f-9e7a-13ab350607a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:10.446293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37c46-0de1-494f-9e7a-13ab350607a9 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:10.446985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e37c46-0de1-494f-9e7a-13ab350607a9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:05:10.447323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e37c46-0de1-494f-9e7a-13ab350607a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:10.448689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e37c46-0de1-494f-9e7a-13ab350607a9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:10.454089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 647/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Tue Aug 2 10:05:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:05:10.459958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:10.570554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fefe-297c-4f5e-86d7-948adfc8c22e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:10.608604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8729b4-33a7-4a01-b72d-f94ae92e590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:10.609219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8729b4-33a7-4a01-b72d-f94ae92e590c None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:10.609501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8729b4-33a7-4a01-b72d-f94ae92e590c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:10.609772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8729b4-33a7-4a01-b72d-f94ae92e590c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:10.610476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8729b4-33a7-4a01-b72d-f94ae92e590c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:10.611905 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8729b4-33a7-4a01-b72d-f94ae92e590c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:10.697810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 659/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:05:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:05:20.450764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=30 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:20.693499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:07.766627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2a48-fd4a-49a7-b60f-e0ea8eb490be None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.767747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2a48-fd4a-49a7-b60f-e0ea8eb490be None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.768295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2a48-fd4a-49a7-b60f-e0ea8eb490be None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.771541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 648/1307] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:07.775479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39e15e-b769-4cec-8d46-8534084f5af6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.776392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39e15e-b769-4cec-8d46-8534084f5af6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.776906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39e15e-b769-4cec-8d46-8534084f5af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.876133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4d39e15e-b769-4cec-8d46-8534084f5af6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:07.878760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4d39e15e-b769-4cec-8d46-8534084f5af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:08.004452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 660/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:08.053348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc244a84-9a8b-402d-bf5a-05732442b4f5 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:08.054232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc244a84-9a8b-402d-bf5a-05732442b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:08.055764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc244a84-9a8b-402d-bf5a-05732442b4f5 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:08.058248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 649/1309] 10.222.0.49 () {54 vars in 965 bytes} [Tue Aug 2 10:12:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:08.066768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416101c9-22c0-4afc-b803-ec3949241f74 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:08.067848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416101c9-22c0-4afc-b803-ec3949241f74 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:08.068690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416101c9-22c0-4afc-b803-ec3949241f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:08.187044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-416101c9-22c0-4afc-b803-ec3949241f74 None None] MFA Rules not processed for user `09cabb184e704aecbb0a3bfb1cd0a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:08.188668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-416101c9-22c0-4afc-b803-ec3949241f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:08.216207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:08.280152 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 661/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Tue Aug 2 10:12:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:12:08.290850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc244a84-9a8b-402d-bf5a-05732442b4f5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:08.293446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cc244a84-9a8b-402d-bf5a-05732442b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:08.440325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc244a84-9a8b-402d-bf5a-05732442b4f5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:08.530132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93857273-b7b9-4ede-b7e6-cffb81484ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:08.530132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857273-b7b9-4ede-b7e6-cffb81484ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:08.530132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857273-b7b9-4ede-b7e6-cffb81484ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:08.530132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857273-b7b9-4ede-b7e6-cffb81484ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:08.531304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-93857273-b7b9-4ede-b7e6-cffb81484ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:08.591179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:08.649812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93857273-b7b9-4ede-b7e6-cffb81484ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:08.651483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93857273-b7b9-4ede-b7e6-cffb81484ce4 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:08.673546 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 650/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:10.999854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f12f-6733-4b3d-9de3-76939b907bdb None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:11.000409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f12f-6733-4b3d-9de3-76939b907bdb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:11.000409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f12f-6733-4b3d-9de3-76939b907bdb None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:11.001803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 662/1312] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:11.007011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7d20a-e7fe-4ecb-8e36-9f29cb0e1506 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:11.007475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7d20a-e7fe-4ecb-8e36-9f29cb0e1506 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:11.007588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7d20a-e7fe-4ecb-8e36-9f29cb0e1506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:11.092951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8dd7d20a-e7fe-4ecb-8e36-9f29cb0e1506 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:11.094193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8dd7d20a-e7fe-4ecb-8e36-9f29cb0e1506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:11.204745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 651/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:11.244535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0f12f-6733-4b3d-9de3-76939b907bdb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:11.246098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-80c0f12f-6733-4b3d-9de3-76939b907bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:11.354376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0f12f-6733-4b3d-9de3-76939b907bdb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:11.427088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:11.430394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9686862-79ac-4ed9-9dc1-6e496c843560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:11.430552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9686862-79ac-4ed9-9dc1-6e496c843560 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:11.430842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9686862-79ac-4ed9-9dc1-6e496c843560 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:11.431121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9686862-79ac-4ed9-9dc1-6e496c843560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:11.432874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c9686862-79ac-4ed9-9dc1-6e496c843560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:11.554354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9686862-79ac-4ed9-9dc1-6e496c843560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:11.555890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9686862-79ac-4ed9-9dc1-6e496c843560 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:11.576466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 663/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:14.178337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e1da-f227-473d-af90-047c68d380f5 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:14.178864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e1da-f227-473d-af90-047c68d380f5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:14.178952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e1da-f227-473d-af90-047c68d380f5 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:14.180810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 652/1315] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:14.186833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4e654-bde1-4127-96fe-70bca005a6ac None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:14.187447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4e654-bde1-4127-96fe-70bca005a6ac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:14.187937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4e654-bde1-4127-96fe-70bca005a6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:14.269908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dac4e654-bde1-4127-96fe-70bca005a6ac None None] MFA Rules not processed for user `bd58a6d89a6c4fa196454cb9055bbb5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:14.271167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dac4e654-bde1-4127-96fe-70bca005a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:14.332532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:14.382126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 664/1316] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:14.436963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab496953-f653-491b-9ee7-32cf55019b9c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:14.437266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab496953-f653-491b-9ee7-32cf55019b9c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:14.437547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab496953-f653-491b-9ee7-32cf55019b9c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:14.438919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 653/1317] 10.222.0.49 () {54 vars in 965 bytes} [Tue Aug 2 10:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:14.444940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3eb12-25b7-4a06-8e76-c545d0bbdd01 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:14.445903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3eb12-25b7-4a06-8e76-c545d0bbdd01 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:14.446223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3eb12-25b7-4a06-8e76-c545d0bbdd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:14.524241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e9f3eb12-25b7-4a06-8e76-c545d0bbdd01 None None] MFA Rules not processed for user `09cabb184e704aecbb0a3bfb1cd0a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:14.525396 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e9f3eb12-25b7-4a06-8e76-c545d0bbdd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:14.604918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 665/1318] 10.222.0.49 () {58 vars in 1062 bytes} [Tue Aug 2 10:12:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:14.614449 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab496953-f653-491b-9ee7-32cf55019b9c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:14.616007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ab496953-f653-491b-9ee7-32cf55019b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:14.626899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:14.725227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab496953-f653-491b-9ee7-32cf55019b9c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:14.815265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203455f-9771-42c4-8609-a6b10704a84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:14.815887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203455f-9771-42c4-8609-a6b10704a84e service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:14.816181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203455f-9771-42c4-8609-a6b10704a84e service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:14.816458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203455f-9771-42c4-8609-a6b10704a84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:14.818103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a203455f-9771-42c4-8609-a6b10704a84e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:14.881512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:14.937930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203455f-9771-42c4-8609-a6b10704a84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:14.939416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203455f-9771-42c4-8609-a6b10704a84e service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:14.958343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 654/1319] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:18.158286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787485a-8065-4547-a60f-6d43693fccb1 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:18.158832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787485a-8065-4547-a60f-6d43693fccb1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:18.158952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787485a-8065-4547-a60f-6d43693fccb1 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:18.160303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 666/1320] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:18.166140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678a36-0339-410f-9c3c-feda65928415 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:18.166576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678a36-0339-410f-9c3c-feda65928415 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:18.166708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678a36-0339-410f-9c3c-feda65928415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:18.238549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7e678a36-0339-410f-9c3c-feda65928415 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:18.239787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7e678a36-0339-410f-9c3c-feda65928415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:18.333643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 655/1321] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:18.374077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787485a-8065-4547-a60f-6d43693fccb1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:18.375723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a787485a-8065-4547-a60f-6d43693fccb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:18.426113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:18.479435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787485a-8065-4547-a60f-6d43693fccb1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:18.556093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073311c5-38c1-4817-8583-bc0acfe64132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:18.556725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073311c5-38c1-4817-8583-bc0acfe64132 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:18.557012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073311c5-38c1-4817-8583-bc0acfe64132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:18.557294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073311c5-38c1-4817-8583-bc0acfe64132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:18.558968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-073311c5-38c1-4817-8583-bc0acfe64132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:18.655840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:18.669745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073311c5-38c1-4817-8583-bc0acfe64132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:18.672289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073311c5-38c1-4817-8583-bc0acfe64132 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:18.689729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 667/1322] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:12:21.756256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a90de9-a146-49dc-a9c5-8e888187f316 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:21.756791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a90de9-a146-49dc-a9c5-8e888187f316 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:21.756902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a90de9-a146-49dc-a9c5-8e888187f316 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:21.758897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 656/1323] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:21.764278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f4367-c2c6-46cd-ad5c-b250d9a09de4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:21.764713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f4367-c2c6-46cd-ad5c-b250d9a09de4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:21.764845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f4367-c2c6-46cd-ad5c-b250d9a09de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:21.830275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-753f4367-c2c6-46cd-ad5c-b250d9a09de4 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:21.831557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-753f4367-c2c6-46cd-ad5c-b250d9a09de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:21.932111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 668/1324] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:21.969640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a90de9-a146-49dc-a9c5-8e888187f316 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:21.971163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-57a90de9-a146-49dc-a9c5-8e888187f316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:21.990977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:22.085377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a90de9-a146-49dc-a9c5-8e888187f316 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:22.167776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788605a-86f7-48db-a54d-89fd24b7e890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:22.168396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788605a-86f7-48db-a54d-89fd24b7e890 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:22.168676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788605a-86f7-48db-a54d-89fd24b7e890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:22.168950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788605a-86f7-48db-a54d-89fd24b7e890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:22.170606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1788605a-86f7-48db-a54d-89fd24b7e890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:22.250107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:22.293477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788605a-86f7-48db-a54d-89fd24b7e890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:22.294835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788605a-86f7-48db-a54d-89fd24b7e890 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:22.313892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 657/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:24.383863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a5a57-17c2-4857-a021-00f4ab99f1e6 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:24.384400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a5a57-17c2-4857-a021-00f4ab99f1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:24.384498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a5a57-17c2-4857-a021-00f4ab99f1e6 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:24.385998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 669/1326] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:24.391406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8d0a4-4db5-41ce-94c3-68513209022b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:24.391844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8d0a4-4db5-41ce-94c3-68513209022b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:24.392109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8d0a4-4db5-41ce-94c3-68513209022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:24.459947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-69e8d0a4-4db5-41ce-94c3-68513209022b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:24.461717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-69e8d0a4-4db5-41ce-94c3-68513209022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:24.562472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 658/1327] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:24.599171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a5a57-17c2-4857-a021-00f4ab99f1e6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:24.600992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7c0a5a57-17c2-4857-a021-00f4ab99f1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:24.663897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:24.708026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a5a57-17c2-4857-a021-00f4ab99f1e6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:24.781152 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5accc9f0-421c-4e24-a7b8-14f1ddf403a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:24.781838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc9f0-421c-4e24-a7b8-14f1ddf403a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:24.782069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc9f0-421c-4e24-a7b8-14f1ddf403a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:24.782351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc9f0-421c-4e24-a7b8-14f1ddf403a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:24.784044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5accc9f0-421c-4e24-a7b8-14f1ddf403a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:24.894420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:24.898558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5accc9f0-421c-4e24-a7b8-14f1ddf403a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:24.900176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5accc9f0-421c-4e24-a7b8-14f1ddf403a2 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:24.923755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 670/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:27.006543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d838-ce70-4959-8943-c133b886246b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.007129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d838-ce70-4959-8943-c133b886246b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.007129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d838-ce70-4959-8943-c133b886246b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.008495 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 659/1329] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:27.014205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef9458-6dfa-482a-bacb-6c99359771a8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.014689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef9458-6dfa-482a-bacb-6c99359771a8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.014798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef9458-6dfa-482a-bacb-6c99359771a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.088924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-11ef9458-6dfa-482a-bacb-6c99359771a8 None None] MFA Rules not processed for user `bd58a6d89a6c4fa196454cb9055bbb5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:27.090254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-11ef9458-6dfa-482a-bacb-6c99359771a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:27.187821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 671/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:12:27.225922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6d838-ce70-4959-8943-c133b886246b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.228231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-12b6d838-ce70-4959-8943-c133b886246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:27.261471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:27.337862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6d838-ce70-4959-8943-c133b886246b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.416211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac4376-39f0-4cea-800e-5285ff3ca2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.416834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac4376-39f0-4cea-800e-5285ff3ca2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.417120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac4376-39f0-4cea-800e-5285ff3ca2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.417396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac4376-39f0-4cea-800e-5285ff3ca2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.419007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e0ac4376-39f0-4cea-800e-5285ff3ca2a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:27.497482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:27.530752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac4376-39f0-4cea-800e-5285ff3ca2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.532529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac4376-39f0-4cea-800e-5285ff3ca2a8 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.552703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 660/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:30.020475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374735-d840-4cde-afb1-23f30a08f842 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:30.021049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374735-d840-4cde-afb1-23f30a08f842 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:30.021049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374735-d840-4cde-afb1-23f30a08f842 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:30.022453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 672/1332] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:30.028106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc09284-ee13-4143-961f-806b3ba0572e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:30.028521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc09284-ee13-4143-961f-806b3ba0572e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:30.028661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc09284-ee13-4143-961f-806b3ba0572e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:30.096838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5cc09284-ee13-4143-961f-806b3ba0572e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:30.098096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5cc09284-ee13-4143-961f-806b3ba0572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:30.194992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 661/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:30.232994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69374735-d840-4cde-afb1-23f30a08f842 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:30.234550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-69374735-d840-4cde-afb1-23f30a08f842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:30.316490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:30.352290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69374735-d840-4cde-afb1-23f30a08f842 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:30.432321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a216c-8c30-4530-9943-649d82775a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:30.432962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a216c-8c30-4530-9943-649d82775a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:30.433253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a216c-8c30-4530-9943-649d82775a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:30.433522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a216c-8c30-4530-9943-649d82775a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:30.435213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b35a216c-8c30-4530-9943-649d82775a4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:30.553134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a216c-8c30-4530-9943-649d82775a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:30.554624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a216c-8c30-4530-9943-649d82775a4c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:30.565048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:30.573992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 673/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:32.641599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999842ab-da45-41ed-8631-df39c32d23bd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:32.642165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999842ab-da45-41ed-8631-df39c32d23bd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:32.642165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999842ab-da45-41ed-8631-df39c32d23bd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:32.643554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 662/1335] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:32.648827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39789c45-6674-4eac-adac-dbf8d9185b57 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:32.649246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39789c45-6674-4eac-adac-dbf8d9185b57 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:32.649391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39789c45-6674-4eac-adac-dbf8d9185b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:32.722473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-39789c45-6674-4eac-adac-dbf8d9185b57 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:32.723761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-39789c45-6674-4eac-adac-dbf8d9185b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:32.828355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 674/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:12:32.869028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999842ab-da45-41ed-8631-df39c32d23bd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:32.870564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-999842ab-da45-41ed-8631-df39c32d23bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:32.911817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:32.979221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999842ab-da45-41ed-8631-df39c32d23bd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:33.055901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779c52b-21d1-4f94-bcbc-69ea420538e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:33.056525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779c52b-21d1-4f94-bcbc-69ea420538e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:33.056810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779c52b-21d1-4f94-bcbc-69ea420538e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:33.057087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779c52b-21d1-4f94-bcbc-69ea420538e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:33.058724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d779c52b-21d1-4f94-bcbc-69ea420538e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:33.144696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:33.172579 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779c52b-21d1-4f94-bcbc-69ea420538e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:33.174083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779c52b-21d1-4f94-bcbc-69ea420538e7 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:33.193264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 663/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:35.925569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320caaa5-2301-485f-9ab0-28f341ec44b4 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:35.926142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320caaa5-2301-485f-9ab0-28f341ec44b4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:35.926142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320caaa5-2301-485f-9ab0-28f341ec44b4 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:35.927534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 675/1338] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:35.933598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a058a6-a817-44db-bbbb-e07030d53309 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:35.934119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a058a6-a817-44db-bbbb-e07030d53309 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:35.934220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a058a6-a817-44db-bbbb-e07030d53309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:36.004812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-09a058a6-a817-44db-bbbb-e07030d53309 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:36.006061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-09a058a6-a817-44db-bbbb-e07030d53309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:36.110007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 664/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:36.146383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320caaa5-2301-485f-9ab0-28f341ec44b4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:36.148077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-320caaa5-2301-485f-9ab0-28f341ec44b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:36.226064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:36.254632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320caaa5-2301-485f-9ab0-28f341ec44b4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:36.327690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49e57f-1d08-4b3b-b7c5-02d414fd8ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:36.328312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49e57f-1d08-4b3b-b7c5-02d414fd8ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:36.328596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49e57f-1d08-4b3b-b7c5-02d414fd8ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:36.328875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49e57f-1d08-4b3b-b7c5-02d414fd8ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:36.330502 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-da49e57f-1d08-4b3b-b7c5-02d414fd8ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:36.437737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49e57f-1d08-4b3b-b7c5-02d414fd8ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:36.439182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49e57f-1d08-4b3b-b7c5-02d414fd8ba6 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:36.457776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 676/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:40.732120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e172b-8f82-42d4-a246-110a978b25f6 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:40.732645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e172b-8f82-42d4-a246-110a978b25f6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:40.732759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e172b-8f82-42d4-a246-110a978b25f6 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:40.734079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 665/1341] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:40.741000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8fca-ffe9-4c32-8942-ca7b6e95f457 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:40.741475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8fca-ffe9-4c32-8942-ca7b6e95f457 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:40.741587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8fca-ffe9-4c32-8942-ca7b6e95f457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:40.747216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:40.812823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-088c8fca-ffe9-4c32-8942-ca7b6e95f457 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:40.814049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-088c8fca-ffe9-4c32-8942-ca7b6e95f457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:40.920752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 677/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:40.963260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e172b-8f82-42d4-a246-110a978b25f6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:40.964798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e31e172b-8f82-42d4-a246-110a978b25f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:41.014118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:41.070950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e172b-8f82-42d4-a246-110a978b25f6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:41.146924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9de8c-077c-47ee-96b7-ee3e60aaee5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:41.147587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9de8c-077c-47ee-96b7-ee3e60aaee5a service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:41.147871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9de8c-077c-47ee-96b7-ee3e60aaee5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:41.148258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9de8c-077c-47ee-96b7-ee3e60aaee5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:41.150585 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-72e9de8c-077c-47ee-96b7-ee3e60aaee5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:41.249656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:41.266834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9de8c-077c-47ee-96b7-ee3e60aaee5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:41.268256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9de8c-077c-47ee-96b7-ee3e60aaee5a service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:41.286986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 666/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:44.300552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d71ea-d302-46d2-9d52-14c3b8d7830a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:44.301111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d71ea-d302-46d2-9d52-14c3b8d7830a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:44.301200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d71ea-d302-46d2-9d52-14c3b8d7830a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:44.302498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 678/1344] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:44.307833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e4e1-df17-46cc-97f9-40dc196d3814 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:44.308286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e4e1-df17-46cc-97f9-40dc196d3814 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:44.308534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1e4e1-df17-46cc-97f9-40dc196d3814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:44.376725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d4a1e4e1-df17-46cc-97f9-40dc196d3814 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:44.377957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d4a1e4e1-df17-46cc-97f9-40dc196d3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:44.493940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 667/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:44.533367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d71ea-d302-46d2-9d52-14c3b8d7830a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:44.534918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f07d71ea-d302-46d2-9d52-14c3b8d7830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:44.624931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:44.642984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d71ea-d302-46d2-9d52-14c3b8d7830a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:44.721559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db7338-958b-4b10-9908-930abd1c0987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:44.722186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7338-958b-4b10-9908-930abd1c0987 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:44.722467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7338-958b-4b10-9908-930abd1c0987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:44.722742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7338-958b-4b10-9908-930abd1c0987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:44.724425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-93db7338-958b-4b10-9908-930abd1c0987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:44.841901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db7338-958b-4b10-9908-930abd1c0987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:44.843423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db7338-958b-4b10-9908-930abd1c0987 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:44.865308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 679/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:49.386538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5a8b0-e6e2-4fe2-899c-af70e37f2a3a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:49.387047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5a8b0-e6e2-4fe2-899c-af70e37f2a3a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:49.387047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5a8b0-e6e2-4fe2-899c-af70e37f2a3a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:49.388443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 668/1347] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:49.393742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925052f-a6ae-4109-b32f-1e6f825b8283 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:49.394162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925052f-a6ae-4109-b32f-1e6f825b8283 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:49.394303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925052f-a6ae-4109-b32f-1e6f825b8283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:49.411069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:49.466750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0925052f-a6ae-4109-b32f-1e6f825b8283 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:49.468079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0925052f-a6ae-4109-b32f-1e6f825b8283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:49.597210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 680/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:12:49.637245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5a8b0-e6e2-4fe2-899c-af70e37f2a3a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:49.638842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-85a5a8b0-e6e2-4fe2-899c-af70e37f2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:49.708142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:49.766237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5a8b0-e6e2-4fe2-899c-af70e37f2a3a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:49.854085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3cb1f-f820-44f3-8eae-0ed4afb17827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:49.854736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3cb1f-f820-44f3-8eae-0ed4afb17827 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:49.855023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3cb1f-f820-44f3-8eae-0ed4afb17827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:49.855362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3cb1f-f820-44f3-8eae-0ed4afb17827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:49.857045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0fc3cb1f-f820-44f3-8eae-0ed4afb17827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:49.973823 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:49.976784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3cb1f-f820-44f3-8eae-0ed4afb17827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:49.978301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3cb1f-f820-44f3-8eae-0ed4afb17827 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:49.997742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 669/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:54.873713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9329-ef57-4a9c-afe3-46cb58d94d26 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:54.874218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9329-ef57-4a9c-afe3-46cb58d94d26 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:54.874314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9329-ef57-4a9c-afe3-46cb58d94d26 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:54.875989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 681/1350] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:12:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:54.882213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d042ec-ab32-4abf-b569-564e6525a327 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:54.883151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d042ec-ab32-4abf-b569-564e6525a327 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:54.883750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d042ec-ab32-4abf-b569-564e6525a327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:54.972806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-33d042ec-ab32-4abf-b569-564e6525a327 None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:54.974970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-33d042ec-ab32-4abf-b569-564e6525a327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:55.183273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 670/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:12:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:12:59.350542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088922-ef95-4a90-8e7c-9e31f9e2c44e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:59.350542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088922-ef95-4a90-8e7c-9e31f9e2c44e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:59.350542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088922-ef95-4a90-8e7c-9e31f9e2c44e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:59.351448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 682/1352] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:12:59.357794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf5c6-34ee-464f-b994-8c651febab9b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:59.358326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf5c6-34ee-464f-b994-8c651febab9b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:59.358588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf5c6-34ee-464f-b994-8c651febab9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:59.434276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:59.464928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1aebf5c6-34ee-464f-b994-8c651febab9b None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:59.466358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1aebf5c6-34ee-464f-b994-8c651febab9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:59.595437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=59 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:59.620052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 671/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:12:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:00.860980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493e76-f157-4b32-bbcc-fcf5ce50f5b0 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:00.861550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493e76-f157-4b32-bbcc-fcf5ce50f5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:00.861550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493e76-f157-4b32-bbcc-fcf5ce50f5b0 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:00.863716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 683/1354] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:00.870263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22952-bd16-4032-b0fe-64ec05ec723a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:00.870774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22952-bd16-4032-b0fe-64ec05ec723a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:00.870869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22952-bd16-4032-b0fe-64ec05ec723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:00.957959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-72b22952-bd16-4032-b0fe-64ec05ec723a None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:00.959205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-72b22952-bd16-4032-b0fe-64ec05ec723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:01.006691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:01.083127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 672/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:13:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:04.711854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83be692-7e39-486b-b55b-abe588c0efb7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:04.712716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83be692-7e39-486b-b55b-abe588c0efb7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:04.713146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83be692-7e39-486b-b55b-abe588c0efb7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:04.716108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 684/1356] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:04.720136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15379a-5494-49c2-8485-39c218263bcd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:04.720639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15379a-5494-49c2-8485-39c218263bcd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:04.720722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15379a-5494-49c2-8485-39c218263bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:04.816666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3e15379a-5494-49c2-8485-39c218263bcd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:04.818159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3e15379a-5494-49c2-8485-39c218263bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:04.935883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 673/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:04.990250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bfffc-01cf-4863-b164-ef686b6b736e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:04.990559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bfffc-01cf-4863-b164-ef686b6b736e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:04.991013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bfffc-01cf-4863-b164-ef686b6b736e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:04.992397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 685/1358] 10.222.0.49 () {54 vars in 962 bytes} [Tue Aug 2 10:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:05.000961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407186e-172e-4b4e-842c-147bf0587def None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:05.001250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407186e-172e-4b4e-842c-147bf0587def None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:05.001528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407186e-172e-4b4e-842c-147bf0587def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:05.023966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:05.103131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a407186e-172e-4b4e-842c-147bf0587def None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:05.104358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a407186e-172e-4b4e-842c-147bf0587def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:05.236723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 674/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Tue Aug 2 10:13:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:05.253089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bfffc-01cf-4863-b164-ef686b6b736e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:05.255018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-094bfffc-01cf-4863-b164-ef686b6b736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:05.423745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bfffc-01cf-4863-b164-ef686b6b736e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:05.506920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:05.553580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fefb23-c651-4171-88dc-5cef5d578f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:05.554211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fefb23-c651-4171-88dc-5cef5d578f69 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:05.554574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fefb23-c651-4171-88dc-5cef5d578f69 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:05.554874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fefb23-c651-4171-88dc-5cef5d578f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:05.556510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d5fefb23-c651-4171-88dc-5cef5d578f69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:05.655668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fefb23-c651-4171-88dc-5cef5d578f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:05.657195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fefb23-c651-4171-88dc-5cef5d578f69 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:05.677857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 686/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:13:07.915428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d0828-6c4e-4be5-b603-4b135ba57643 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:07.915977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d0828-6c4e-4be5-b603-4b135ba57643 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:07.915977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d0828-6c4e-4be5-b603-4b135ba57643 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:07.922593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c645c3f-1ba4-45a5-a5ad-a73926c441a6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:07.923111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c645c3f-1ba4-45a5-a5ad-a73926c441a6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:07.923391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c645c3f-1ba4-45a5-a5ad-a73926c441a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:07.927057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 675/1361] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:07.994568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7c645c3f-1ba4-45a5-a5ad-a73926c441a6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:07.995827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7c645c3f-1ba4-45a5-a5ad-a73926c441a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:08.034900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:08.096107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 687/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:13:08.141881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d0828-6c4e-4be5-b603-4b135ba57643 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:08.143644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-be5d0828-6c4e-4be5-b603-4b135ba57643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:08.243984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:08.351821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d0828-6c4e-4be5-b603-4b135ba57643 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:08.490059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab90f1-a482-4fc6-a769-923fc05697a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:08.490742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab90f1-a482-4fc6-a769-923fc05697a6 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:08.491017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab90f1-a482-4fc6-a769-923fc05697a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:08.491331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab90f1-a482-4fc6-a769-923fc05697a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:08.493145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-90ab90f1-a482-4fc6-a769-923fc05697a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:08.522768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:08.603298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab90f1-a482-4fc6-a769-923fc05697a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:08.605435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab90f1-a482-4fc6-a769-923fc05697a6 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:08.631519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 676/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:11.148741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2414c-f49e-475e-b751-4337bcca46d3 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:11.149247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2414c-f49e-475e-b751-4337bcca46d3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:11.149247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2414c-f49e-475e-b751-4337bcca46d3 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:11.150587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 688/1364] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:11.156376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04070-ceee-4a59-a434-054c0e308788 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:11.157324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04070-ceee-4a59-a434-054c0e308788 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:11.157769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04070-ceee-4a59-a434-054c0e308788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:11.241026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-02f04070-ceee-4a59-a434-054c0e308788 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:11.259100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-02f04070-ceee-4a59-a434-054c0e308788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:11.361882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:11.367267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 677/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:11.415228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5024a-b514-4b90-82df-5371e9d5174d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:11.415548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5024a-b514-4b90-82df-5371e9d5174d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:11.415810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5024a-b514-4b90-82df-5371e9d5174d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:11.417348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 689/1366] 10.222.0.49 () {54 vars in 962 bytes} [Tue Aug 2 10:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:11.423650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e677d-dc94-464d-9087-889e7d39108a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:11.424069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e677d-dc94-464d-9087-889e7d39108a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:11.424391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e677d-dc94-464d-9087-889e7d39108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:11.509439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f35e677d-dc94-464d-9087-889e7d39108a None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:11.509439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f35e677d-dc94-464d-9087-889e7d39108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:11.644831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 678/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Tue Aug 2 10:13:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:11.653098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5024a-b514-4b90-82df-5371e9d5174d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:11.654815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9b5024a-b514-4b90-82df-5371e9d5174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:11.816065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5024a-b514-4b90-82df-5371e9d5174d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:11.839026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:11.943736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5c8c-f420-4553-8e41-ea39c09925fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:11.944276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5c8c-f420-4553-8e41-ea39c09925fe service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:11.944558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5c8c-f420-4553-8e41-ea39c09925fe service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:11.944834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5c8c-f420-4553-8e41-ea39c09925fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:11.947082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a6db5c8c-f420-4553-8e41-ea39c09925fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:12.042627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db5c8c-f420-4553-8e41-ea39c09925fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:12.044276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db5c8c-f420-4553-8e41-ea39c09925fe service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:12.064418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 690/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:13.118873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f868-31cb-420e-a8fd-66a60f3f4b41 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:13.119503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f868-31cb-420e-a8fd-66a60f3f4b41 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:13.119503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f868-31cb-420e-a8fd-66a60f3f4b41 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:13.121003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 679/1369] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:13.126144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad024-5c29-411b-bc41-7b6c0a87aee9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:13.126689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad024-5c29-411b-bc41-7b6c0a87aee9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:13.127124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad024-5c29-411b-bc41-7b6c0a87aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:13.173534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:13.211386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c21ad024-5c29-411b-bc41-7b6c0a87aee9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:13.212756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c21ad024-5c29-411b-bc41-7b6c0a87aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:13.317496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 691/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:13.640014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbebe52-3bdb-4be6-854f-7204df805b14 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:13.640312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbebe52-3bdb-4be6-854f-7204df805b14 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:13.640597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbebe52-3bdb-4be6-854f-7204df805b14 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:13.642193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 680/1371] 10.222.0.49 () {54 vars in 964 bytes} [Tue Aug 2 10:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:13.647130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15dc11-6195-40db-a52a-16fe5e24ad18 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:13.647673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15dc11-6195-40db-a52a-16fe5e24ad18 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:13.647975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15dc11-6195-40db-a52a-16fe5e24ad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:13.731563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fd15dc11-6195-40db-a52a-16fe5e24ad18 None None] MFA Rules not processed for user `2dbb92bcc5054287a96fbfac03d2975a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:13.733040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fd15dc11-6195-40db-a52a-16fe5e24ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:13.782409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:13.813769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 692/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Tue Aug 2 10:13:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:13.822102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbebe52-3bdb-4be6-854f-7204df805b14 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:13.823967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ddbebe52-3bdb-4be6-854f-7204df805b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:13.882820 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:13.947785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbebe52-3bdb-4be6-854f-7204df805b14 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:14.022614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1b441-f3eb-4809-8d03-6d80dd39c0c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:14.023291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1b441-f3eb-4809-8d03-6d80dd39c0c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:14.023575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1b441-f3eb-4809-8d03-6d80dd39c0c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:14.023849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1b441-f3eb-4809-8d03-6d80dd39c0c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:14.025460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-76d1b441-f3eb-4809-8d03-6d80dd39c0c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:14.127317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:14.144203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1b441-f3eb-4809-8d03-6d80dd39c0c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:14.145839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1b441-f3eb-4809-8d03-6d80dd39c0c0 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:14.167945 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 681/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:14.372096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097090a6-270f-4cdb-956a-f848bef0c863 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:14.372525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097090a6-270f-4cdb-956a-f848bef0c863 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:14.372706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097090a6-270f-4cdb-956a-f848bef0c863 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:14.374725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 693/1374] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:14.380150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6c019-efc5-460a-a885-5c069213571d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:14.380444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6c019-efc5-460a-a885-5c069213571d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:14.380703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6c019-efc5-460a-a885-5c069213571d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:14.453106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1da6c019-efc5-460a-a885-5c069213571d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:14.454439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1da6c019-efc5-460a-a885-5c069213571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:14.565933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 682/1375] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:14.599688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097090a6-270f-4cdb-956a-f848bef0c863 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:14.601217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-097090a6-270f-4cdb-956a-f848bef0c863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:14.764876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097090a6-270f-4cdb-956a-f848bef0c863 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:14.817630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:14.880848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4e977-bfe2-431d-bd72-1830d7ce74f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:14.881406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4e977-bfe2-431d-bd72-1830d7ce74f5 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:14.881673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4e977-bfe2-431d-bd72-1830d7ce74f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:14.881950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4e977-bfe2-431d-bd72-1830d7ce74f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:14.884236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dbf4e977-bfe2-431d-bd72-1830d7ce74f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:14.978639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4e977-bfe2-431d-bd72-1830d7ce74f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:14.980272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4e977-bfe2-431d-bd72-1830d7ce74f5 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:15.005981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 694/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:17.242695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65696-f0dd-4c77-b85c-3e838ae9d527 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:17.243247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65696-f0dd-4c77-b85c-3e838ae9d527 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:17.243344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65696-f0dd-4c77-b85c-3e838ae9d527 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:17.244735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 683/1377] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:17.250275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b91a3-0c0a-44f5-96ae-c83ad33db068 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:17.251077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b91a3-0c0a-44f5-96ae-c83ad33db068 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:17.251403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b91a3-0c0a-44f5-96ae-c83ad33db068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:17.327908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6e4b91a3-0c0a-44f5-96ae-c83ad33db068 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:17.329115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6e4b91a3-0c0a-44f5-96ae-c83ad33db068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:17.336430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:17.441896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 695/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:17.486907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f65696-f0dd-4c77-b85c-3e838ae9d527 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:17.488748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-83f65696-f0dd-4c77-b85c-3e838ae9d527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:17.557293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:17.652508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f65696-f0dd-4c77-b85c-3e838ae9d527 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:17.786623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1af31-4bee-49f5-910e-b741c190a045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:17.787388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1af31-4bee-49f5-910e-b741c190a045 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:17.787744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1af31-4bee-49f5-910e-b741c190a045 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:17.788090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1af31-4bee-49f5-910e-b741c190a045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:17.789822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e2a1af31-4bee-49f5-910e-b741c190a045 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:17.814301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:17.911860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1af31-4bee-49f5-910e-b741c190a045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:17.914072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1af31-4bee-49f5-910e-b741c190a045 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:17.938295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 684/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:20.121284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca7c3a-b58d-4f02-86c0-0b7160592fb8 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:20.121818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca7c3a-b58d-4f02-86c0-0b7160592fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:20.121818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca7c3a-b58d-4f02-86c0-0b7160592fb8 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:20.123352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 696/1380] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:20.128485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7642a-c747-4d1b-bd7d-e9b653289f98 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:20.128901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7642a-c747-4d1b-bd7d-e9b653289f98 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:20.129057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7642a-c747-4d1b-bd7d-e9b653289f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:20.202232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-03a7642a-c747-4d1b-bd7d-e9b653289f98 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:20.203460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-03a7642a-c747-4d1b-bd7d-e9b653289f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:20.310393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:20.316681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 685/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:20.360739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca7c3a-b58d-4f02-86c0-0b7160592fb8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:20.362364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9eca7c3a-b58d-4f02-86c0-0b7160592fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:20.517450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:20.531789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca7c3a-b58d-4f02-86c0-0b7160592fb8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:20.654369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af9c93-da0e-49b6-991f-71a532b03ced service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:20.654992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af9c93-da0e-49b6-991f-71a532b03ced service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:20.655303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af9c93-da0e-49b6-991f-71a532b03ced service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:20.655582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af9c93-da0e-49b6-991f-71a532b03ced service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:20.657202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-42af9c93-da0e-49b6-991f-71a532b03ced service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:20.764115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af9c93-da0e-49b6-991f-71a532b03ced service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:20.765778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af9c93-da0e-49b6-991f-71a532b03ced service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:20.778570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:20.787203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 697/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:13:23.022899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df7d5-e746-43ba-95a1-b357b87064ff None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:23.023498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df7d5-e746-43ba-95a1-b357b87064ff None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:23.023498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df7d5-e746-43ba-95a1-b357b87064ff None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:23.024897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 686/1383] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:23.030021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a976-9e87-4391-b96e-7972d146860e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:23.030021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a976-9e87-4391-b96e-7972d146860e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:23.031163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a976-9e87-4391-b96e-7972d146860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:23.100007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7861a976-9e87-4391-b96e-7972d146860e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:23.101263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7861a976-9e87-4391-b96e-7972d146860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:23.223725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 698/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:23.276569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df7d5-e746-43ba-95a1-b357b87064ff None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:23.278437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9d2df7d5-e746-43ba-95a1-b357b87064ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:23.460637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df7d5-e746-43ba-95a1-b357b87064ff None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:23.547153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:23.596298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c65e1f-48af-4e22-8d2a-1fa6adb46c2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:23.596879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65e1f-48af-4e22-8d2a-1fa6adb46c2a service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:23.597163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65e1f-48af-4e22-8d2a-1fa6adb46c2a service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:23.597440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65e1f-48af-4e22-8d2a-1fa6adb46c2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:23.599094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c3c65e1f-48af-4e22-8d2a-1fa6adb46c2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:23.694164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c65e1f-48af-4e22-8d2a-1fa6adb46c2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:23.695894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c65e1f-48af-4e22-8d2a-1fa6adb46c2a service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:23.716602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 687/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:25.791470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd3261-386c-426a-876a-e975783097dd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:25.791990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd3261-386c-426a-876a-e975783097dd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:25.791990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd3261-386c-426a-876a-e975783097dd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:25.793410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 699/1386] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:25.798761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba32b9-e2c8-4338-bdd8-4115ada6fd6e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:25.799298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba32b9-e2c8-4338-bdd8-4115ada6fd6e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:25.799456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba32b9-e2c8-4338-bdd8-4115ada6fd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:25.869239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7eba32b9-e2c8-4338-bdd8-4115ada6fd6e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:25.871033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7eba32b9-e2c8-4338-bdd8-4115ada6fd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:25.908624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:25.979143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 688/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:26.024168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd3261-386c-426a-876a-e975783097dd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:26.026207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d2dd3261-386c-426a-876a-e975783097dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:26.111855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:26.201933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd3261-386c-426a-876a-e975783097dd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:26.329012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01fd4-2f56-46e8-999c-4f6051fd65f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:26.329805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01fd4-2f56-46e8-999c-4f6051fd65f8 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:26.330231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01fd4-2f56-46e8-999c-4f6051fd65f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:26.330587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01fd4-2f56-46e8-999c-4f6051fd65f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:26.332723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2dd01fd4-2f56-46e8-999c-4f6051fd65f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:26.382501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:26.441279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd01fd4-2f56-46e8-999c-4f6051fd65f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:26.442982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd01fd4-2f56-46e8-999c-4f6051fd65f8 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:26.461942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 700/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:28.938936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedb461-aa2b-4eaf-abc8-247a1c8470dd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:28.939521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedb461-aa2b-4eaf-abc8-247a1c8470dd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:28.939521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedb461-aa2b-4eaf-abc8-247a1c8470dd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:28.940854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 689/1389] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:28.946752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce165-33b0-473d-9961-07e304592612 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:28.947228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce165-33b0-473d-9961-07e304592612 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:28.947365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce165-33b0-473d-9961-07e304592612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:29.014784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8cdce165-33b0-473d-9961-07e304592612 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:29.016049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8cdce165-33b0-473d-9961-07e304592612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:29.124067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 701/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:29.168917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedb461-aa2b-4eaf-abc8-247a1c8470dd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:29.170626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-efedb461-aa2b-4eaf-abc8-247a1c8470dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:29.325776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedb461-aa2b-4eaf-abc8-247a1c8470dd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:29.342087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:29.443983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a948a-7604-4b5a-a78d-459bd38cdb3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:29.444608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a948a-7604-4b5a-a78d-459bd38cdb3a service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:29.444889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a948a-7604-4b5a-a78d-459bd38cdb3a service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:29.445162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a948a-7604-4b5a-a78d-459bd38cdb3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:29.446860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3b9a948a-7604-4b5a-a78d-459bd38cdb3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:29.541608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a948a-7604-4b5a-a78d-459bd38cdb3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:29.543263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a948a-7604-4b5a-a78d-459bd38cdb3a service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:29.561852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 690/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:30.198475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e716f88-050d-41c5-ab74-2c7c95095361 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:30.199087 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e716f88-050d-41c5-ab74-2c7c95095361 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:30.199087 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e716f88-050d-41c5-ab74-2c7c95095361 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:30.200897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 702/1392] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:30.206413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ab7e-0ead-4e3e-b305-ce9bca6305e8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:30.206870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ab7e-0ead-4e3e-b305-ce9bca6305e8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:30.206974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ab7e-0ead-4e3e-b305-ce9bca6305e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:30.233234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:30.275489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9c81ab7e-0ead-4e3e-b305-ce9bca6305e8 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:30.276727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9c81ab7e-0ead-4e3e-b305-ce9bca6305e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:30.375053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 691/1393] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:30.434153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d531aa-0c4d-468e-ae4a-a751072a0841 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:30.434532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d531aa-0c4d-468e-ae4a-a751072a0841 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:30.434875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d531aa-0c4d-468e-ae4a-a751072a0841 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:30.436323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 703/1394] 10.222.0.49 () {54 vars in 966 bytes} [Tue Aug 2 10:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:30.441868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19f0d-ecdc-4d1e-b467-12a8e342a6f1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:30.442300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19f0d-ecdc-4d1e-b467-12a8e342a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:30.442594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19f0d-ecdc-4d1e-b467-12a8e342a6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:30.519625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9be19f0d-ecdc-4d1e-b467-12a8e342a6f1 None None] MFA Rules not processed for user `700720d6f66e4de2a65b808afce70e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:30.520835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9be19f0d-ecdc-4d1e-b467-12a8e342a6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:30.563199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:30.600136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 692/1395] 10.222.0.49 () {58 vars in 1063 bytes} [Tue Aug 2 10:13:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:30.608444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d531aa-0c4d-468e-ae4a-a751072a0841 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:30.609944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-72d531aa-0c4d-468e-ae4a-a751072a0841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:30.623652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:30.717577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d531aa-0c4d-468e-ae4a-a751072a0841 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:30.798410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f26a-84ef-4226-83b2-7fb86efb8603 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:30.799052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f26a-84ef-4226-83b2-7fb86efb8603 service glance] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:30.799375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f26a-84ef-4226-83b2-7fb86efb8603 service glance] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:30.799653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f26a-84ef-4226-83b2-7fb86efb8603 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:30.801372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-92d1f26a-84ef-4226-83b2-7fb86efb8603 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:30.886030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:30.928995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f26a-84ef-4226-83b2-7fb86efb8603 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:30.930469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f26a-84ef-4226-83b2-7fb86efb8603 service glance] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:30.952501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 704/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:31.786543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cf2d6-cd4c-40e4-ac4b-63859252de0f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:31.787172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cf2d6-cd4c-40e4-ac4b-63859252de0f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:31.787172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cf2d6-cd4c-40e4-ac4b-63859252de0f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:31.788567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 693/1397] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:31.795454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:31.796038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:31.796142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:31.874523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:31.876135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:31.882971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae87f4-0273-4fed-bbaf-23493350f510 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:31.883531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae87f4-0273-4fed-bbaf-23493350f510 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:31.883901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae87f4-0273-4fed-bbaf-23493350f510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:31.953391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f8ae87f4-0273-4fed-bbaf-23493350f510 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:31.954627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f8ae87f4-0273-4fed-bbaf-23493350f510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:31.983442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 705/1398] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:32.031544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:32.032823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:32.053680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 694/1399] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:13:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:13:32.060427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:32.110240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68321bd-2c17-4859-ba79-abfd0a2460b5 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:32.110240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68321bd-2c17-4859-ba79-abfd0a2460b5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:32.110240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68321bd-2c17-4859-ba79-abfd0a2460b5 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:32.110240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 695/1400] 10.222.0.49 () {54 vars in 943 bytes} [Tue Aug 2 10:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:13:32.110240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5d26-afa5-42b1-a38b-2cd898673fb7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:32.110240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5d26-afa5-42b1-a38b-2cd898673fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:32.112234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5d26-afa5-42b1-a38b-2cd898673fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:32.159826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de63fb-b374-488d-bc06-81418a897324 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:32.184006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:32.191629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-daaf5d26-afa5-42b1-a38b-2cd898673fb7 None None] MFA Rules not processed for user `66f7db7eea94472c82cda44c569d9550`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:32.192991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-daaf5d26-afa5-42b1-a38b-2cd898673fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:32.285065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef697a90-1cfb-411d-8ad8-312cfca13b14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:32.285702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef697a90-1cfb-411d-8ad8-312cfca13b14 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:32.285988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef697a90-1cfb-411d-8ad8-312cfca13b14 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:32.286267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef697a90-1cfb-411d-8ad8-312cfca13b14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:32.288002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ef697a90-1cfb-411d-8ad8-312cfca13b14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:32.305183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 696/1401] 10.222.0.49 () {58 vars in 1040 bytes} [Tue Aug 2 10:13:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:13:32.314622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5d26-afa5-42b1-a38b-2cd898673fb7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:32.324864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5d26-afa5-42b1-a38b-2cd898673fb7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:32.346055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:32.383774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef697a90-1cfb-411d-8ad8-312cfca13b14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:32.385762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef697a90-1cfb-411d-8ad8-312cfca13b14 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:32.410537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 706/1402] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:13:32.437980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac2169-6086-452d-8528-e46b4cf31223 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:32.438772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac2169-6086-452d-8528-e46b4cf31223 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:32.439059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac2169-6086-452d-8528-e46b4cf31223 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:32.439369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac2169-6086-452d-8528-e46b4cf31223 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:32.441098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1dac2169-6086-452d-8528-e46b4cf31223 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:32.444052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:32.532646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac2169-6086-452d-8528-e46b4cf31223 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:32.534232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac2169-6086-452d-8528-e46b4cf31223 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:32.542564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 697/1403] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:13:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:13:34.598734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568a265-37f4-4dfa-b811-35d85d9c77f1 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:34.599313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568a265-37f4-4dfa-b811-35d85d9c77f1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:34.599313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568a265-37f4-4dfa-b811-35d85d9c77f1 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:34.602260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 707/1404] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:34.606295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49282a4-3d29-4399-8d39-85152ad03889 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:34.606838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49282a4-3d29-4399-8d39-85152ad03889 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:34.606838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49282a4-3d29-4399-8d39-85152ad03889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:34.685187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f49282a4-3d29-4399-8d39-85152ad03889 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:34.686443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f49282a4-3d29-4399-8d39-85152ad03889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:34.801786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:34.808419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 698/1405] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:34.855037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568a265-37f4-4dfa-b811-35d85d9c77f1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:34.856690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e568a265-37f4-4dfa-b811-35d85d9c77f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:35.025846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:35.028920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568a265-37f4-4dfa-b811-35d85d9c77f1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:35.156839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8d77d-1faa-46d4-8ff4-f3db5ef70c30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:35.157496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8d77d-1faa-46d4-8ff4-f3db5ef70c30 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:35.157860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8d77d-1faa-46d4-8ff4-f3db5ef70c30 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:35.158214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8d77d-1faa-46d4-8ff4-f3db5ef70c30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:35.159991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3fb8d77d-1faa-46d4-8ff4-f3db5ef70c30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:35.263363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8d77d-1faa-46d4-8ff4-f3db5ef70c30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:35.265042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8d77d-1faa-46d4-8ff4-f3db5ef70c30 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:35.288608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 708/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:37.531159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8288bc0-0e63-42cf-ad3d-8475ad5750fb None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:37.531159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8288bc0-0e63-42cf-ad3d-8475ad5750fb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:37.531682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8288bc0-0e63-42cf-ad3d-8475ad5750fb None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:37.538221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64544267-d3c8-4583-aa16-782e566cab2f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:37.538664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64544267-d3c8-4583-aa16-782e566cab2f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:37.538896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64544267-d3c8-4583-aa16-782e566cab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:37.544237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:37.544377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 699/1407] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:13:37 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:13:37.612004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-64544267-d3c8-4583-aa16-782e566cab2f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:37.613280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-64544267-d3c8-4583-aa16-782e566cab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:37.745558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 709/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:37.790311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8288bc0-0e63-42cf-ad3d-8475ad5750fb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:37.791937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a8288bc0-0e63-42cf-ad3d-8475ad5750fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:38.006378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8288bc0-0e63-42cf-ad3d-8475ad5750fb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:38.090667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:38.150535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc848a-9756-459b-b16b-8d503820f410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:38.151170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc848a-9756-459b-b16b-8d503820f410 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:38.151488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc848a-9756-459b-b16b-8d503820f410 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:38.151768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc848a-9756-459b-b16b-8d503820f410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:38.153628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6ddc848a-9756-459b-b16b-8d503820f410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:38.269643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc848a-9756-459b-b16b-8d503820f410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:38.271346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc848a-9756-459b-b16b-8d503820f410 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:38.292924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 700/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:48.286680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=64 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:48.385966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=64 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:56.693088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f98d-d44b-46ab-8897-dceb3e6ef888 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:56.693630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f98d-d44b-46ab-8897-dceb3e6ef888 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:56.693630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f98d-d44b-46ab-8897-dceb3e6ef888 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:56.694991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 710/1410] 10.222.0.49 () {54 vars in 918 bytes} [Tue Aug 2 10:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:56.700237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19e0b3-54c5-4b5d-8220-cd6c6a65546c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:56.700697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19e0b3-54c5-4b5d-8220-cd6c6a65546c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:56.700864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19e0b3-54c5-4b5d-8220-cd6c6a65546c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:56.778865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1e19e0b3-54c5-4b5d-8220-cd6c6a65546c None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:56.780113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1e19e0b3-54c5-4b5d-8220-cd6c6a65546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:56.886076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 701/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Tue Aug 2 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:56.900866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc8425-5b5f-414e-a9bf-40f9b75d7b6c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:56.901153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc8425-5b5f-414e-a9bf-40f9b75d7b6c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:56.901429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc8425-5b5f-414e-a9bf-40f9b75d7b6c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:56.902792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 711/1412] 10.222.0.49 () {54 vars in 918 bytes} [Tue Aug 2 10:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:56.907669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fa78d-f74e-4b87-9de9-a9b32401c2fa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:56.907964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fa78d-f74e-4b87-9de9-a9b32401c2fa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:56.908248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fa78d-f74e-4b87-9de9-a9b32401c2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:56.983313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:56.986429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a60fa78d-f74e-4b87-9de9-a9b32401c2fa None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:56.987696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a60fa78d-f74e-4b87-9de9-a9b32401c2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:57.104609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 702/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Tue Aug 2 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:59.743141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219454ca-411a-45aa-924f-ca96943b2fcd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:59.744186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219454ca-411a-45aa-924f-ca96943b2fcd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:59.744318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219454ca-411a-45aa-924f-ca96943b2fcd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:59.746541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:13:59.752155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dccccc-91f2-447c-a9af-5ac8c3f3d5a6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:59.752653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dccccc-91f2-447c-a9af-5ac8c3f3d5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:59.752762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dccccc-91f2-447c-a9af-5ac8c3f3d5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:59.819725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a7dccccc-91f2-447c-a9af-5ac8c3f3d5a6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:13:59.820921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a7dccccc-91f2-447c-a9af-5ac8c3f3d5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:59.885695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:59.926561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:13:59.969632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219454ca-411a-45aa-924f-ca96943b2fcd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:59.971164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-219454ca-411a-45aa-924f-ca96943b2fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:00.118553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219454ca-411a-45aa-924f-ca96943b2fcd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:00.192359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:00.233968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358784b-6171-429c-b7cc-2e496a423c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:00.234620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358784b-6171-429c-b7cc-2e496a423c24 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:00.234906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358784b-6171-429c-b7cc-2e496a423c24 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:00.235213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358784b-6171-429c-b7cc-2e496a423c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:00.236847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7358784b-6171-429c-b7cc-2e496a423c24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:00.331462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358784b-6171-429c-b7cc-2e496a423c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:00.333073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358784b-6171-429c-b7cc-2e496a423c24 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:00.351054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 713/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:01.257129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622368e-a967-4e91-918a-271f57b4083e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:01.257722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622368e-a967-4e91-918a-271f57b4083e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:01.257722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622368e-a967-4e91-918a-271f57b4083e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:01.259052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:01.264071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ff94-c226-4b11-8133-39c81d483dae None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:01.264499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ff94-c226-4b11-8133-39c81d483dae None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:01.264633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ff94-c226-4b11-8133-39c81d483dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:01.344986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f9f0ff94-c226-4b11-8133-39c81d483dae None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:01.346185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f9f0ff94-c226-4b11-8133-39c81d483dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:01.373235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:01.441743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:01.592787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622368e-a967-4e91-918a-271f57b4083e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:01.594312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d622368e-a967-4e91-918a-271f57b4083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:01.738672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622368e-a967-4e91-918a-271f57b4083e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:01.786849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:01.860315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a637a2-7b67-4c4e-9453-b92d5c6cd68a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:01.860950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a637a2-7b67-4c4e-9453-b92d5c6cd68a service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:01.861238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a637a2-7b67-4c4e-9453-b92d5c6cd68a service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:01.861518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a637a2-7b67-4c4e-9453-b92d5c6cd68a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:01.863207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-19a637a2-7b67-4c4e-9453-b92d5c6cd68a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:01.956658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a637a2-7b67-4c4e-9453-b92d5c6cd68a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:01.958277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a637a2-7b67-4c4e-9453-b92d5c6cd68a service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:01.976340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 705/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:02.863108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a79b-ca3a-4fb0-98c8-ecbc6a0a9aa3 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:02.863690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a79b-ca3a-4fb0-98c8-ecbc6a0a9aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:02.863690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a79b-ca3a-4fb0-98c8-ecbc6a0a9aa3 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:02.865039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:02.870828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e291a-5d7b-48ca-96bc-b07b283d7ee1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:02.871263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e291a-5d7b-48ca-96bc-b07b283d7ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:02.871451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e291a-5d7b-48ca-96bc-b07b283d7ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:02.926988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:02.949474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8c1e291a-5d7b-48ca-96bc-b07b283d7ee1 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:02.950743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8c1e291a-5d7b-48ca-96bc-b07b283d7ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:03.050310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:03.205669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2a79b-ca3a-4fb0-98c8-ecbc6a0a9aa3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:03.207358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-74b2a79b-ca3a-4fb0-98c8-ecbc6a0a9aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:03.354163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2a79b-ca3a-4fb0-98c8-ecbc6a0a9aa3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:03.366532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:03.469170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54e3a6-99b0-4823-8a6f-5857713e2cbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:03.469711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e3a6-99b0-4823-8a6f-5857713e2cbc service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:03.469995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e3a6-99b0-4823-8a6f-5857713e2cbc service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:03.470275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e3a6-99b0-4823-8a6f-5857713e2cbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:03.472016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4f54e3a6-99b0-4823-8a6f-5857713e2cbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:03.567768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54e3a6-99b0-4823-8a6f-5857713e2cbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:03.569570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54e3a6-99b0-4823-8a6f-5857713e2cbc service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:03.588853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 716/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:04.457075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0bbb-61d0-4b82-a477-ef99a063ce1d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:04.457688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0bbb-61d0-4b82-a477-ef99a063ce1d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:04.457688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0bbb-61d0-4b82-a477-ef99a063ce1d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:04.459040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:04.465304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908c7fa-3cdb-4ae4-816f-c87e4b964d8f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:04.465859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908c7fa-3cdb-4ae4-816f-c87e4b964d8f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:04.465956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908c7fa-3cdb-4ae4-816f-c87e4b964d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:04.487103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:04.545410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5908c7fa-3cdb-4ae4-816f-c87e4b964d8f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:04.546619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5908c7fa-3cdb-4ae4-816f-c87e4b964d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:04.640544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:04.791555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa0bbb-61d0-4b82-a477-ef99a063ce1d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:04.793936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-84fa0bbb-61d0-4b82-a477-ef99a063ce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:04.904921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:04.941223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa0bbb-61d0-4b82-a477-ef99a063ce1d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:05.058899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f445b0-1a4c-4d49-8633-380ba47f8979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:05.059473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f445b0-1a4c-4d49-8633-380ba47f8979 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:05.059763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f445b0-1a4c-4d49-8633-380ba47f8979 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:05.060043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f445b0-1a4c-4d49-8633-380ba47f8979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:05.061667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c7f445b0-1a4c-4d49-8633-380ba47f8979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:05.138161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:05.151994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f445b0-1a4c-4d49-8633-380ba47f8979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:05.153631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f445b0-1a4c-4d49-8633-380ba47f8979 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:05.171559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 708/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:06.038023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ba6c9-ca3d-42f9-b89d-1d24893aad76 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:06.038587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ba6c9-ca3d-42f9-b89d-1d24893aad76 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:06.038587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ba6c9-ca3d-42f9-b89d-1d24893aad76 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:06.039999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:06.044968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2f80-b6c7-4c2f-a947-33f76c7c965c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:06.045403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2f80-b6c7-4c2f-a947-33f76c7c965c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:06.045605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2f80-b6c7-4c2f-a947-33f76c7c965c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:06.119971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9dac2f80-b6c7-4c2f-a947-33f76c7c965c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:06.131784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9dac2f80-b6c7-4c2f-a947-33f76c7c965c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:06.220013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:06.370719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ba6c9-ca3d-42f9-b89d-1d24893aad76 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:06.372323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c66ba6c9-ca3d-42f9-b89d-1d24893aad76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:06.455474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:06.516647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ba6c9-ca3d-42f9-b89d-1d24893aad76 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:06.630338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97819288-917c-4ef1-80ea-f9dc7446e64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:06.630893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819288-917c-4ef1-80ea-f9dc7446e64e service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:06.631178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819288-917c-4ef1-80ea-f9dc7446e64e service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:06.631493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819288-917c-4ef1-80ea-f9dc7446e64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:06.633160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-97819288-917c-4ef1-80ea-f9dc7446e64e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:06.680788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:06.721968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97819288-917c-4ef1-80ea-f9dc7446e64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:06.723616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97819288-917c-4ef1-80ea-f9dc7446e64e service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:06.742130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 719/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:07.645156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de97620-0a37-467d-b6ff-f5e0e6bbd2f4 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:07.645717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de97620-0a37-467d-b6ff-f5e0e6bbd2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:07.645717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de97620-0a37-467d-b6ff-f5e0e6bbd2f4 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:07.647066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:07.652866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8781-e2a9-43d4-8eca-0191168b0a19 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:07.653304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8781-e2a9-43d4-8eca-0191168b0a19 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:07.653412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8781-e2a9-43d4-8eca-0191168b0a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:07.728883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-296b8781-e2a9-43d4-8eca-0191168b0a19 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:07.730108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-296b8781-e2a9-43d4-8eca-0191168b0a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:07.827590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:07.984332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de97620-0a37-467d-b6ff-f5e0e6bbd2f4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:07.985853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6de97620-0a37-467d-b6ff-f5e0e6bbd2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:08.035159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:08.131222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de97620-0a37-467d-b6ff-f5e0e6bbd2f4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:08.246628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad23f8-fedb-4cc1-b457-671e3deaf8d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:08.247289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad23f8-fedb-4cc1-b457-671e3deaf8d8 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:08.247578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad23f8-fedb-4cc1-b457-671e3deaf8d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:08.247858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad23f8-fedb-4cc1-b457-671e3deaf8d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:08.249496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ffad23f8-fedb-4cc1-b457-671e3deaf8d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:08.256500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:08.339375 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad23f8-fedb-4cc1-b457-671e3deaf8d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:08.341743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad23f8-fedb-4cc1-b457-671e3deaf8d8 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:08.361141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 711/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:09.232146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09996e2-195a-4e60-92d0-1ceb4983bee9 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:09.232779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09996e2-195a-4e60-92d0-1ceb4983bee9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:09.232779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09996e2-195a-4e60-92d0-1ceb4983bee9 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:09.234160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:09.239180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7b92f-de3c-490c-886b-c4276a48fb31 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:09.239635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7b92f-de3c-490c-886b-c4276a48fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:09.239768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7b92f-de3c-490c-886b-c4276a48fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:09.317345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d7a7b92f-de3c-490c-886b-c4276a48fb31 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:09.318563 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d7a7b92f-de3c-490c-886b-c4276a48fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:09.386350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:09.416825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:09.570492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09996e2-195a-4e60-92d0-1ceb4983bee9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:09.572712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c09996e2-195a-4e60-92d0-1ceb4983bee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:09.590076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:09.721717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09996e2-195a-4e60-92d0-1ceb4983bee9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:09.816385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:09.842441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362dbbd-5615-4138-8931-9f4cb9ce72f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:09.843065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362dbbd-5615-4138-8931-9f4cb9ce72f1 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:09.843385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362dbbd-5615-4138-8931-9f4cb9ce72f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:09.843665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362dbbd-5615-4138-8931-9f4cb9ce72f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:09.845308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9362dbbd-5615-4138-8931-9f4cb9ce72f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:09.938576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362dbbd-5615-4138-8931-9f4cb9ce72f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:09.940568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362dbbd-5615-4138-8931-9f4cb9ce72f1 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:09.959222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 722/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:10.845989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7440a2-ad9e-459f-b2a8-c4df3c83abad None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:10.846513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7440a2-ad9e-459f-b2a8-c4df3c83abad None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:10.846513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7440a2-ad9e-459f-b2a8-c4df3c83abad None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:10.847861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:10.852953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974199d-7351-438a-b82c-38d4eada4128 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:10.853388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974199d-7351-438a-b82c-38d4eada4128 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:10.853573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974199d-7351-438a-b82c-38d4eada4128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:10.931922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8974199d-7351-438a-b82c-38d4eada4128 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:10.933139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8974199d-7351-438a-b82c-38d4eada4128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:10.974610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:11.031665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:11.182584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7440a2-ad9e-459f-b2a8-c4df3c83abad None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:11.184029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8a7440a2-ad9e-459f-b2a8-c4df3c83abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:11.328919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7440a2-ad9e-459f-b2a8-c4df3c83abad None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:11.389408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:11.449260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075669-77f1-4a75-b84d-32da43f013ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:11.449809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075669-77f1-4a75-b84d-32da43f013ac service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:11.450096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075669-77f1-4a75-b84d-32da43f013ac service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:11.450376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075669-77f1-4a75-b84d-32da43f013ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:11.452027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-77075669-77f1-4a75-b84d-32da43f013ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:11.541035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77075669-77f1-4a75-b84d-32da43f013ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:11.542647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77075669-77f1-4a75-b84d-32da43f013ac service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:11.560570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 714/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:12.449464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7639025-9951-4277-a281-08df73782803 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:12.449975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7639025-9951-4277-a281-08df73782803 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:12.449975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7639025-9951-4277-a281-08df73782803 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:12.451311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:12.456330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aef92-3ca3-4979-a69a-cdd1877734e2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:12.456780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aef92-3ca3-4979-a69a-cdd1877734e2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:12.456880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aef92-3ca3-4979-a69a-cdd1877734e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:12.534887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-510aef92-3ca3-4979-a69a-cdd1877734e2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:12.536159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-510aef92-3ca3-4979-a69a-cdd1877734e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:12.538266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:12.639109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:12.787441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7639025-9951-4277-a281-08df73782803 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:12.789023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c7639025-9951-4277-a281-08df73782803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:12.937507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7639025-9951-4277-a281-08df73782803 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:12.970060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:13.061226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6f3ac-7b1d-4c23-8842-5ffae512ebe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:13.061759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f3ac-7b1d-4c23-8842-5ffae512ebe2 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:13.062043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f3ac-7b1d-4c23-8842-5ffae512ebe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:13.062318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f3ac-7b1d-4c23-8842-5ffae512ebe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:13.063975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ebc6f3ac-7b1d-4c23-8842-5ffae512ebe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:13.156595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6f3ac-7b1d-4c23-8842-5ffae512ebe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:13.158210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6f3ac-7b1d-4c23-8842-5ffae512ebe2 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:13.177973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 725/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:15.088123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4043-758b-4636-8e02-8798475add7a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:15.088845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4043-758b-4636-8e02-8798475add7a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:15.088845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4043-758b-4636-8e02-8798475add7a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:15.090070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:15.096410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f19ea-f217-46b5-8d70-5031e380d1a9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:15.096895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f19ea-f217-46b5-8d70-5031e380d1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:15.097394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f19ea-f217-46b5-8d70-5031e380d1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:15.141451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:15.290197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ce2f19ea-f217-46b5-8d70-5031e380d1a9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:15.291507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ce2f19ea-f217-46b5-8d70-5031e380d1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:15.401100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:15.440231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4043-758b-4636-8e02-8798475add7a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:15.441868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-292a4043-758b-4636-8e02-8798475add7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:15.550141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4043-758b-4636-8e02-8798475add7a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:15.583105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:15.623943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7a591-babb-4af2-bf35-7c8157f47491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:15.624572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a591-babb-4af2-bf35-7c8157f47491 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:15.624854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a591-babb-4af2-bf35-7c8157f47491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:15.625136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a591-babb-4af2-bf35-7c8157f47491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:15.627583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bbc7a591-babb-4af2-bf35-7c8157f47491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:15.738615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7a591-babb-4af2-bf35-7c8157f47491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:15.740081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7a591-babb-4af2-bf35-7c8157f47491 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:15.767959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 717/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:17.922524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5305e-6844-48a0-83fe-d8dea4e577fb None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:17.923067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5305e-6844-48a0-83fe-d8dea4e577fb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:17.923067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5305e-6844-48a0-83fe-d8dea4e577fb None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:17.924383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:17.929586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc73cf-d454-479d-b299-d228422cd89b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:17.929992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc73cf-d454-479d-b299-d228422cd89b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:17.930148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc73cf-d454-479d-b299-d228422cd89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:18.003242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cfdc73cf-d454-479d-b299-d228422cd89b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:18.004475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cfdc73cf-d454-479d-b299-d228422cd89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:18.017149 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:18.223852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:18.261918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5305e-6844-48a0-83fe-d8dea4e577fb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:18.263528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4da5305e-6844-48a0-83fe-d8dea4e577fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:18.370116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5305e-6844-48a0-83fe-d8dea4e577fb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:18.382563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:18.443346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d97d4c-1fc6-4baf-82b3-be82c1f79192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:18.443971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d97d4c-1fc6-4baf-82b3-be82c1f79192 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:18.444253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d97d4c-1fc6-4baf-82b3-be82c1f79192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:18.444527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d97d4c-1fc6-4baf-82b3-be82c1f79192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:18.446148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c5d97d4c-1fc6-4baf-82b3-be82c1f79192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:18.554312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d97d4c-1fc6-4baf-82b3-be82c1f79192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:18.555776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d97d4c-1fc6-4baf-82b3-be82c1f79192 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:18.574711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 728/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:20.931261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb141e-ad83-4093-908a-620e9915ea32 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:20.931819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb141e-ad83-4093-908a-620e9915ea32 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:20.931819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb141e-ad83-4093-908a-620e9915ea32 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:20.933117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:20.938586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a44d6-0f14-4df8-b205-90ab173a5beb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:20.939007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a44d6-0f14-4df8-b205-90ab173a5beb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:20.939142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a44d6-0f14-4df8-b205-90ab173a5beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:20.987376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:21.010487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-380a44d6-0f14-4df8-b205-90ab173a5beb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:21.011755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-380a44d6-0f14-4df8-b205-90ab173a5beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:21.106842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:21.145724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb141e-ad83-4093-908a-620e9915ea32 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:21.152795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bafb141e-ad83-4093-908a-620e9915ea32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:21.252249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb141e-ad83-4093-908a-620e9915ea32 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:21.291871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:21.326005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bbb3c-6760-49a4-998a-0552c12f6544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:21.326628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bbb3c-6760-49a4-998a-0552c12f6544 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:21.326915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bbb3c-6760-49a4-998a-0552c12f6544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:21.327211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bbb3c-6760-49a4-998a-0552c12f6544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:21.328849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-055bbb3c-6760-49a4-998a-0552c12f6544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:21.444665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bbb3c-6760-49a4-998a-0552c12f6544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:21.446107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bbb3c-6760-49a4-998a-0552c12f6544 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:21.464573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 720/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:23.592330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba20a3-e54a-4445-a914-75a694865964 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:23.592818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba20a3-e54a-4445-a914-75a694865964 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:23.592922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba20a3-e54a-4445-a914-75a694865964 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:23.594301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:23.599467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4080-6587-4924-9821-8ef1b5623b8f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:23.599878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4080-6587-4924-9821-8ef1b5623b8f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:23.600026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4080-6587-4924-9821-8ef1b5623b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:23.665281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-223c4080-6587-4924-9821-8ef1b5623b8f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:23.666519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-223c4080-6587-4924-9821-8ef1b5623b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:23.687170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:23.764648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:23.803431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba20a3-e54a-4445-a914-75a694865964 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:23.804966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-42ba20a3-e54a-4445-a914-75a694865964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:23.907575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba20a3-e54a-4445-a914-75a694865964 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:23.927039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:23.982830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9fde1-a04d-4351-9574-6a8ade4d22cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:23.983528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9fde1-a04d-4351-9574-6a8ade4d22cb service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:23.990626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9fde1-a04d-4351-9574-6a8ade4d22cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:23.990626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9fde1-a04d-4351-9574-6a8ade4d22cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:23.990626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-96d9fde1-a04d-4351-9574-6a8ade4d22cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:24.096185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9fde1-a04d-4351-9574-6a8ade4d22cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:24.097530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9fde1-a04d-4351-9574-6a8ade4d22cb service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:24.115962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 731/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:26.251356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472ff7-777c-4f4e-9186-2c56a49d7966 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:26.251939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472ff7-777c-4f4e-9186-2c56a49d7966 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:26.251939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472ff7-777c-4f4e-9186-2c56a49d7966 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:26.253252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:26.258609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45039864-391f-485a-af1f-2649fa30bd25 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:26.259040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45039864-391f-485a-af1f-2649fa30bd25 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:26.259158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45039864-391f-485a-af1f-2649fa30bd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:26.322651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:26.325511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-45039864-391f-485a-af1f-2649fa30bd25 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:26.326734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-45039864-391f-485a-af1f-2649fa30bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:26.425512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:14:26.463436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472ff7-777c-4f4e-9186-2c56a49d7966 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:26.464939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-77472ff7-777c-4f4e-9186-2c56a49d7966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:26.570006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472ff7-777c-4f4e-9186-2c56a49d7966 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:26.624332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:26.648996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9ed86-a82a-4067-b7c5-e87ad3e3e1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:26.649620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ed86-a82a-4067-b7c5-e87ad3e3e1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:26.649903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ed86-a82a-4067-b7c5-e87ad3e3e1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:26.650181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ed86-a82a-4067-b7c5-e87ad3e3e1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:26.652370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dde9ed86-a82a-4067-b7c5-e87ad3e3e1f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:26.760907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9ed86-a82a-4067-b7c5-e87ad3e3e1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:26.762350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9ed86-a82a-4067-b7c5-e87ad3e3e1f7 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:26.781100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 723/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:28.835926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b5e99-a523-4234-9f95-13c7b0d6c2d5 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:28.836447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b5e99-a523-4234-9f95-13c7b0d6c2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:28.836447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b5e99-a523-4234-9f95-13c7b0d6c2d5 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:28.837784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:28.844345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d7bc2-dbe3-461a-a7a6-d632f1acdc09 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:28.844899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d7bc2-dbe3-461a-a7a6-d632f1acdc09 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:28.844899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d7bc2-dbe3-461a-a7a6-d632f1acdc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:28.913355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1d1d7bc2-dbe3-461a-a7a6-d632f1acdc09 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:28.914588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1d1d7bc2-dbe3-461a-a7a6-d632f1acdc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:28.944096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:29.012511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:29.049554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b5e99-a523-4234-9f95-13c7b0d6c2d5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:29.051145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d03b5e99-a523-4234-9f95-13c7b0d6c2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:29.160185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b5e99-a523-4234-9f95-13c7b0d6c2d5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:29.189795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:29.237929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e54e40-e6b3-4053-aa48-a8c1d6e21f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:29.238545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54e40-e6b3-4053-aa48-a8c1d6e21f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:29.238823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54e40-e6b3-4053-aa48-a8c1d6e21f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:29.239096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54e40-e6b3-4053-aa48-a8c1d6e21f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:29.240737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-47e54e40-e6b3-4053-aa48-a8c1d6e21f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:29.352730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54e40-e6b3-4053-aa48-a8c1d6e21f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:29.355305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54e40-e6b3-4053-aa48-a8c1d6e21f93 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:29.375056 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 734/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:31.475642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43f8e8-9153-4011-b84f-8db141cb53ee None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:31.476218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43f8e8-9153-4011-b84f-8db141cb53ee None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:31.476218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43f8e8-9153-4011-b84f-8db141cb53ee None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:31.477626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:31.483324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c2901-8c16-40a2-b3da-58cbbc073d2e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:31.483772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c2901-8c16-40a2-b3da-58cbbc073d2e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:31.483872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c2901-8c16-40a2-b3da-58cbbc073d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:31.550787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1d8c2901-8c16-40a2-b3da-58cbbc073d2e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:31.552044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1d8c2901-8c16-40a2-b3da-58cbbc073d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:31.561015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:31.653476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:31.690908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43f8e8-9153-4011-b84f-8db141cb53ee None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:31.692480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bb43f8e8-9153-4011-b84f-8db141cb53ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:31.796220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43f8e8-9153-4011-b84f-8db141cb53ee None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:31.850950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:31.869779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c3d0d-5cf6-4a72-a506-1e7be7f3feea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:31.870397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3d0d-5cf6-4a72-a506-1e7be7f3feea service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:31.870674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3d0d-5cf6-4a72-a506-1e7be7f3feea service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:31.870941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3d0d-5cf6-4a72-a506-1e7be7f3feea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:31.872576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-381c3d0d-5cf6-4a72-a506-1e7be7f3feea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:31.978617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c3d0d-5cf6-4a72-a506-1e7be7f3feea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:31.980574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c3d0d-5cf6-4a72-a506-1e7be7f3feea service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:31.998430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 726/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:34.054322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172166-5910-447a-8e74-18262e23c23f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:34.054916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172166-5910-447a-8e74-18262e23c23f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:34.054916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172166-5910-447a-8e74-18262e23c23f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:34.056289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:34.062418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e812f45-0063-40bf-ab12-5045cf9b1504 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:34.062849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e812f45-0063-40bf-ab12-5045cf9b1504 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:34.062967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e812f45-0063-40bf-ab12-5045cf9b1504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:34.129942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6e812f45-0063-40bf-ab12-5045cf9b1504 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:34.131161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6e812f45-0063-40bf-ab12-5045cf9b1504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:34.171211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:34.246183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:34.286105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05172166-5910-447a-8e74-18262e23c23f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:34.287746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-05172166-5910-447a-8e74-18262e23c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:34.393623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05172166-5910-447a-8e74-18262e23c23f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:34.434402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:34.472701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f1658-5d5b-43a7-b1d6-a50aba73ee17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:34.473328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f1658-5d5b-43a7-b1d6-a50aba73ee17 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:34.473609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f1658-5d5b-43a7-b1d6-a50aba73ee17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:34.473882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f1658-5d5b-43a7-b1d6-a50aba73ee17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:34.475541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7f7f1658-5d5b-43a7-b1d6-a50aba73ee17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:34.594833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f1658-5d5b-43a7-b1d6-a50aba73ee17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:34.596311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f1658-5d5b-43a7-b1d6-a50aba73ee17 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:34.615663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 737/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:36.713233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748382f7-4a36-45ba-b06d-aedb65265d18 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:36.714106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748382f7-4a36-45ba-b06d-aedb65265d18 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:36.714106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748382f7-4a36-45ba-b06d-aedb65265d18 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:36.715336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:36.721556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebefc2-7ec1-4ecc-8da4-d35e05570dc7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:36.722097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebefc2-7ec1-4ecc-8da4-d35e05570dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:36.722097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebefc2-7ec1-4ecc-8da4-d35e05570dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:36.790027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7cebefc2-7ec1-4ecc-8da4-d35e05570dc7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:36.791665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7cebefc2-7ec1-4ecc-8da4-d35e05570dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:36.809781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:36.901022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:36.938293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748382f7-4a36-45ba-b06d-aedb65265d18 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:36.940356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-748382f7-4a36-45ba-b06d-aedb65265d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:37.052031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748382f7-4a36-45ba-b06d-aedb65265d18 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:37.126304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:37.133767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad1c3b-8bcf-4fbc-94cb-3ac4f2001164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:37.134382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1c3b-8bcf-4fbc-94cb-3ac4f2001164 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:37.134670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1c3b-8bcf-4fbc-94cb-3ac4f2001164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:37.134945 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1c3b-8bcf-4fbc-94cb-3ac4f2001164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:37.137228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e9ad1c3b-8bcf-4fbc-94cb-3ac4f2001164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:37.258107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad1c3b-8bcf-4fbc-94cb-3ac4f2001164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:37.259604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad1c3b-8bcf-4fbc-94cb-3ac4f2001164 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:37.279813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 729/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:14:39.315717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661eb4c-98a5-4124-a90e-3c94b4f57b3d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:39.316303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661eb4c-98a5-4124-a90e-3c94b4f57b3d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:39.316303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661eb4c-98a5-4124-a90e-3c94b4f57b3d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:39.317656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:39.323074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c34228-3e54-42ef-83a7-afccc234ace5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:39.323515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c34228-3e54-42ef-83a7-afccc234ace5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:39.323673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c34228-3e54-42ef-83a7-afccc234ace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:39.391938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-90c34228-3e54-42ef-83a7-afccc234ace5 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:39.393191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-90c34228-3e54-42ef-83a7-afccc234ace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:39.441970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:39.490676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:39.527940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661eb4c-98a5-4124-a90e-3c94b4f57b3d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:39.529532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5661eb4c-98a5-4124-a90e-3c94b4f57b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:39.635122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661eb4c-98a5-4124-a90e-3c94b4f57b3d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:39.684584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:39.710423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41f7be-a3c3-4308-b9ac-72d52aafd051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:39.711045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41f7be-a3c3-4308-b9ac-72d52aafd051 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:39.711364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41f7be-a3c3-4308-b9ac-72d52aafd051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:39.711671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41f7be-a3c3-4308-b9ac-72d52aafd051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:39.713851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-de41f7be-a3c3-4308-b9ac-72d52aafd051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:39.824398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41f7be-a3c3-4308-b9ac-72d52aafd051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:39.825768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41f7be-a3c3-4308-b9ac-72d52aafd051 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:39.845260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 740/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:14:49.488814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=22 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:49.842727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=66 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:59.559994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77f7b-ac33-4ada-8ac0-23b3d3a8882b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:59.560496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77f7b-ac33-4ada-8ac0-23b3d3a8882b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:59.560605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77f7b-ac33-4ada-8ac0-23b3d3a8882b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:59.561903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 731/1471] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:59.567037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892dbd4-efa3-46ed-98ca-2eb21996626f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:59.567470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892dbd4-efa3-46ed-98ca-2eb21996626f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:59.567645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892dbd4-efa3-46ed-98ca-2eb21996626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:59.633913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c892dbd4-efa3-46ed-98ca-2eb21996626f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:59.635121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c892dbd4-efa3-46ed-98ca-2eb21996626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:59.735129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 741/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:59.823809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd28a-d405-47f3-8a3a-c4817fa68a88 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:59.824196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd28a-d405-47f3-8a3a-c4817fa68a88 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:59.824422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd28a-d405-47f3-8a3a-c4817fa68a88 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:59.825804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 732/1473] 10.222.0.49 () {54 vars in 966 bytes} [Tue Aug 2 10:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:14:59.830810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdc7e4-4cb7-42fd-b424-2785b7e261c0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:59.831101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdc7e4-4cb7-42fd-b424-2785b7e261c0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:59.831439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdc7e4-4cb7-42fd-b424-2785b7e261c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:59.910356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6bfdc7e4-4cb7-42fd-b424-2785b7e261c0 None None] MFA Rules not processed for user `700720d6f66e4de2a65b808afce70e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:14:59.911614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6bfdc7e4-4cb7-42fd-b424-2785b7e261c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:59.932774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:59.989252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 742/1474] 10.222.0.49 () {58 vars in 1063 bytes} [Tue Aug 2 10:14:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:14:59.997994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cd28a-d405-47f3-8a3a-c4817fa68a88 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:59.999541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0f5cd28a-d405-47f3-8a3a-c4817fa68a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:00.115604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cd28a-d405-47f3-8a3a-c4817fa68a88 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:00.192150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4e6bd-fd61-421d-a1cd-8b40fd98ffb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:00.192781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4e6bd-fd61-421d-a1cd-8b40fd98ffb1 service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:00.193063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4e6bd-fd61-421d-a1cd-8b40fd98ffb1 service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:00.193337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4e6bd-fd61-421d-a1cd-8b40fd98ffb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:00.194985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b3a4e6bd-fd61-421d-a1cd-8b40fd98ffb1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:00.254705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:00.310622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4e6bd-fd61-421d-a1cd-8b40fd98ffb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:00.312118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4e6bd-fd61-421d-a1cd-8b40fd98ffb1 service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:00.329974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 733/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:02.477576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a72345-9938-4c42-93af-5d46b1130d4f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:02.479099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a72345-9938-4c42-93af-5d46b1130d4f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:02.479752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a72345-9938-4c42-93af-5d46b1130d4f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:02.481373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 743/1476] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:15:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:15:02.486985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60177d-bdcd-44fe-815f-218fb87da18c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:02.487503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60177d-bdcd-44fe-815f-218fb87da18c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:02.487750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60177d-bdcd-44fe-815f-218fb87da18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:02.578786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3f60177d-bdcd-44fe-815f-218fb87da18c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:02.580203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3f60177d-bdcd-44fe-815f-218fb87da18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:02.682049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 734/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:15:02.706010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a72345-9938-4c42-93af-5d46b1130d4f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:02.707644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-06a72345-9938-4c42-93af-5d46b1130d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:02.832745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a72345-9938-4c42-93af-5d46b1130d4f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:02.891151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:02.936079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6665047-c875-4f7b-815e-7538f0315adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:02.936704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6665047-c875-4f7b-815e-7538f0315adc admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:02.936989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6665047-c875-4f7b-815e-7538f0315adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:02.937263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6665047-c875-4f7b-815e-7538f0315adc admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:02.937955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6665047-c875-4f7b-815e-7538f0315adc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:02.939723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-d6665047-c875-4f7b-815e-7538f0315adc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:15:02.939723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:15:02.939723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:15:02.940881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6665047-c875-4f7b-815e-7538f0315adc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:02.960571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 744/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Tue Aug 2 10:15:02 2022] GET /identity/v3/projects => generated 2553 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:15:04.989506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e2ff-2ba9-48a9-ad6c-e3b5c20a9c72 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:04.990071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e2ff-2ba9-48a9-ad6c-e3b5c20a9c72 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:04.990071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e2ff-2ba9-48a9-ad6c-e3b5c20a9c72 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:04.991483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 735/1479] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:15:04.997161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d921-bb0c-4ba7-b049-018fee14f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:04.997568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d921-bb0c-4ba7-b049-018fee14f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:04.997747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d921-bb0c-4ba7-b049-018fee14f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:05.062209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-47c2d921-bb0c-4ba7-b049-018fee14f7d2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:05.063447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-47c2d921-bb0c-4ba7-b049-018fee14f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:05.161911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 745/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:05.205341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5e2ff-2ba9-48a9-ad6c-e3b5c20a9c72 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:05.206852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-84b5e2ff-2ba9-48a9-ad6c-e3b5c20a9c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:05.217014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:05.355665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5e2ff-2ba9-48a9-ad6c-e3b5c20a9c72 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:05.436606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:05.469012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90209d08-cbdb-482a-9498-5f0a329f6e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:05.469628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209d08-cbdb-482a-9498-5f0a329f6e06 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:05.469907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209d08-cbdb-482a-9498-5f0a329f6e06 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:05.470184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90209d08-cbdb-482a-9498-5f0a329f6e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:05.471917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-90209d08-cbdb-482a-9498-5f0a329f6e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:05.563148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90209d08-cbdb-482a-9498-5f0a329f6e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:05.564808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90209d08-cbdb-482a-9498-5f0a329f6e06 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:05.582708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 736/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:07.762934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76efe38-6432-458d-8e7f-67f52bfd7626 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:07.763846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76efe38-6432-458d-8e7f-67f52bfd7626 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:07.763846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76efe38-6432-458d-8e7f-67f52bfd7626 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:07.764982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 746/1482] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:15:07.770045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ba490-8a6c-4e3f-8869-b5a1d435ee05 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:07.770443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ba490-8a6c-4e3f-8869-b5a1d435ee05 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:07.770598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ba490-8a6c-4e3f-8869-b5a1d435ee05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:07.835629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6e6ba490-8a6c-4e3f-8869-b5a1d435ee05 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:07.836729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6e6ba490-8a6c-4e3f-8869-b5a1d435ee05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:07.882468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:07.931375 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 737/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:07.967809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76efe38-6432-458d-8e7f-67f52bfd7626 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:07.969345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d76efe38-6432-458d-8e7f-67f52bfd7626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:07.992585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:08.083277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76efe38-6432-458d-8e7f-67f52bfd7626 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:08.154852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1623f71-920d-44dc-989b-802fd6e8e343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:08.155425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623f71-920d-44dc-989b-802fd6e8e343 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:08.155710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623f71-920d-44dc-989b-802fd6e8e343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:08.155983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623f71-920d-44dc-989b-802fd6e8e343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:08.157584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d1623f71-920d-44dc-989b-802fd6e8e343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:08.229273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:08.264696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1623f71-920d-44dc-989b-802fd6e8e343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:08.266125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1623f71-920d-44dc-989b-802fd6e8e343 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:08.283831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 747/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:10.343797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bafd64-c6ea-4c48-ab5c-5d8b8ea4c5fb None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:10.344369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bafd64-c6ea-4c48-ab5c-5d8b8ea4c5fb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:10.344369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bafd64-c6ea-4c48-ab5c-5d8b8ea4c5fb None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:10.345728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 738/1485] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:15:10.351095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671903-e3f8-40c4-8325-2dd6293efc80 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:10.351561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671903-e3f8-40c4-8325-2dd6293efc80 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:10.351698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671903-e3f8-40c4-8325-2dd6293efc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:10.418813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-be671903-e3f8-40c4-8325-2dd6293efc80 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:10.420060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-be671903-e3f8-40c4-8325-2dd6293efc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:10.516340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 748/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:10.554473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bafd64-c6ea-4c48-ab5c-5d8b8ea4c5fb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:10.556091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-11bafd64-c6ea-4c48-ab5c-5d8b8ea4c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:10.659077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bafd64-c6ea-4c48-ab5c-5d8b8ea4c5fb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:10.735691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:10.737465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea462027-7b9e-4d62-af71-4a533c5dddd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:10.738082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462027-7b9e-4d62-af71-4a533c5dddd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:10.738367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462027-7b9e-4d62-af71-4a533c5dddd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:10.738645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462027-7b9e-4d62-af71-4a533c5dddd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:10.740301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ea462027-7b9e-4d62-af71-4a533c5dddd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:10.849661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea462027-7b9e-4d62-af71-4a533c5dddd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:10.851105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea462027-7b9e-4d62-af71-4a533c5dddd7 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:10.870631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 739/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:13.019384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6202f-e245-4c95-83e8-cbfcfb00571d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:13.019939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6202f-e245-4c95-83e8-cbfcfb00571d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:13.019939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6202f-e245-4c95-83e8-cbfcfb00571d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:13.021332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 749/1488] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:15:13.026704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261378a1-f16e-4ff4-a048-0a716f90d053 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:13.027137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261378a1-f16e-4ff4-a048-0a716f90d053 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:13.027298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261378a1-f16e-4ff4-a048-0a716f90d053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:13.093318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-261378a1-f16e-4ff4-a048-0a716f90d053 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:13.094536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-261378a1-f16e-4ff4-a048-0a716f90d053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:13.146834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:13.191088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 740/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:15:13.232149 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6202f-e245-4c95-83e8-cbfcfb00571d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:13.233815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aef6202f-e245-4c95-83e8-cbfcfb00571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:13.270846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:13.339432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6202f-e245-4c95-83e8-cbfcfb00571d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:13.412669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8778980-96e4-4a20-88a8-a2ff0818840f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:13.413283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8778980-96e4-4a20-88a8-a2ff0818840f service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:13.413563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8778980-96e4-4a20-88a8-a2ff0818840f service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:13.413836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8778980-96e4-4a20-88a8-a2ff0818840f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:13.415457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c8778980-96e4-4a20-88a8-a2ff0818840f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:13.493339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:13.522851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8778980-96e4-4a20-88a8-a2ff0818840f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:13.524257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8778980-96e4-4a20-88a8-a2ff0818840f service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:13.542634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 750/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:15:15.891606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21912125-ebe3-4959-aa85-50ba86e7dab6 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:15.892155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21912125-ebe3-4959-aa85-50ba86e7dab6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:15.892155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21912125-ebe3-4959-aa85-50ba86e7dab6 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:15.893426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 741/1491] 10.222.0.49 () {54 vars in 926 bytes} [Tue Aug 2 10:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:15:15.898611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801004b-6d81-4015-83ff-57e83af2663f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:15.899044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801004b-6d81-4015-83ff-57e83af2663f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:15.899238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801004b-6d81-4015-83ff-57e83af2663f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:15.965393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0801004b-6d81-4015-83ff-57e83af2663f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:15:15.967031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0801004b-6d81-4015-83ff-57e83af2663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:16.065493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 751/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Tue Aug 2 10:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:15:16.104282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21912125-ebe3-4959-aa85-50ba86e7dab6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:16.106482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-21912125-ebe3-4959-aa85-50ba86e7dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:16.209030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21912125-ebe3-4959-aa85-50ba86e7dab6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:16.280463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022effed-2e04-455c-b032-979ae5f97c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:16.281080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022effed-2e04-455c-b032-979ae5f97c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:16.281361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022effed-2e04-455c-b032-979ae5f97c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:16.281635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022effed-2e04-455c-b032-979ae5f97c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:16.283257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-022effed-2e04-455c-b032-979ae5f97c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:16.292712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:16.391119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022effed-2e04-455c-b032-979ae5f97c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:16.392601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022effed-2e04-455c-b032-979ae5f97c70 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:16.411267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 742/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:26.063637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=82 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:26.409115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=50 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:00.837825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:00.838391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:00.838391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:00.915043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:00.916290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:01.012446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 752/1494] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:01.021241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a952a-0111-4139-b993-65630f9f59eb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.021744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a952a-0111-4139-b993-65630f9f59eb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.021846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a952a-0111-4139-b993-65630f9f59eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.099025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8a8a952a-0111-4139-b993-65630f9f59eb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:01.100180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8a8a952a-0111-4139-b993-65630f9f59eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:01.195881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 743/1495] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:01.200518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.201792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:01.213161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54875441-e335-4b32-80ca-0671fb0a7eea None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:01.273151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:01.307158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:01.307804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.308089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.308371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.309064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:17:01.309383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.310375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:01.310375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:17:01.310375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:17:01.311299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.323366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 753/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:17:01.331430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a952a-0111-4139-b993-65630f9f59eb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.340792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a952a-0111-4139-b993-65630f9f59eb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:01.402510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:01.436391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:01.437019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.437300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.438193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.438919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.440769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:01.440769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: warnings.warn(msg) Aug 02 10:17:01.440769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: Aug 02 10:17:01.442033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.617752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 744/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:17:01 2022] POST /identity/v3/projects => generated 379 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:01.622156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.623781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:01.657716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1811a08-ea11-415a-b2b2-5f8a776b2561 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:01.751656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:01.752276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:01.752563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:01.752837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:01.753504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:01.754503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:01.754503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:17:01.754503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:17:01.755747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:01.867597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 754/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:17:01 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:17:01.871654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:01.873254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:01.940962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1e9ed-b370-468d-8efe-8b27c9aaaa70 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:02.027545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:02.038126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:02.042819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.042819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.042819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.042819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:02.042819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:02.042819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: warnings.warn(msg) Aug 02 10:17:02.042819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: Aug 02 10:17:02.043436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:02.055526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 745/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:02.059622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:02.061127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:02.073083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba3ff6-79f1-4631-b443-c201eb954903 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:02.097461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:02.169133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4d4a0-ef2d-4a1c-8125-431db65a4fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:02.169786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4d4a0-ef2d-4a1c-8125-431db65a4fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.170071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4d4a0-ef2d-4a1c-8125-431db65a4fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.170350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4d4a0-ef2d-4a1c-8125-431db65a4fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.171041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4d4a0-ef2d-4a1c-8125-431db65a4fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:02.172590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-0bf4d4a0-ef2d-4a1c-8125-431db65a4fe8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:02.172590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:17:02.172590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:17:02.173475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4d4a0-ef2d-4a1c-8125-431db65a4fe8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:02.185818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 755/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:17:02.191140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:02.192693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:02.225676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3612d-ac73-4b7d-b028-16d0d515f207 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:02.324518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153579c-d31c-48ad-b9f3-af2bf55a7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:02.325189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153579c-d31c-48ad-b9f3-af2bf55a7b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.325474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153579c-d31c-48ad-b9f3-af2bf55a7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.325826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153579c-d31c-48ad-b9f3-af2bf55a7b9b admin admin] PATH_INFO: `/v3/projects/1dcb251d85fa486596bc3dda8a99df4e/users/1a8e0b1bad8d4162a4791cf1fecfc114/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.347756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153579c-d31c-48ad-b9f3-af2bf55a7b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dcb251d85fa486596bc3dda8a99df4e, user_id=1a8e0b1bad8d4162a4791cf1fecfc114, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:02.348783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-c153579c-d31c-48ad-b9f3-af2bf55a7b9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:02.348783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: warnings.warn(msg) Aug 02 10:17:02.348783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: Aug 02 10:17:02.350778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153579c-d31c-48ad-b9f3-af2bf55a7b9b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:02.432126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 746/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:17:02 2022] PUT /identity/v3/projects/1dcb251d85fa486596bc3dda8a99df4e/users/1a8e0b1bad8d4162a4791cf1fecfc114/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:02.444654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6499b-0d8e-43ee-9d27-ab197ca11057 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.444906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6499b-0d8e-43ee-9d27-ab197ca11057 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.445208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6499b-0d8e-43ee-9d27-ab197ca11057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.509233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-66c6499b-0d8e-43ee-9d27-ab197ca11057 None None] MFA Rules not processed for user `1a8e0b1bad8d4162a4791cf1fecfc114`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:02.510471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-66c6499b-0d8e-43ee-9d27-ab197ca11057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:02.589018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 756/1502] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:02.603064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da589d0-1d43-4333-a01d-646f98046dad None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.603387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da589d0-1d43-4333-a01d-646f98046dad None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.603680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da589d0-1d43-4333-a01d-646f98046dad None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.605040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 747/1503] 10.222.0.49 () {54 vars in 964 bytes} [Tue Aug 2 10:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:17:02.610028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a57f7-a4b0-49a4-bd2b-d14e97956c26 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.610841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a57f7-a4b0-49a4-bd2b-d14e97956c26 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.611124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a57f7-a4b0-49a4-bd2b-d14e97956c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.643249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:02.691461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-382a57f7-a4b0-49a4-bd2b-d14e97956c26 None None] MFA Rules not processed for user `2dbb92bcc5054287a96fbfac03d2975a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:02.692773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-382a57f7-a4b0-49a4-bd2b-d14e97956c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:02.770512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 757/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Tue Aug 2 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:02.778453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da589d0-1d43-4333-a01d-646f98046dad None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:02.780025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8da589d0-1d43-4333-a01d-646f98046dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:02.832743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:02.890937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da589d0-1d43-4333-a01d-646f98046dad None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:02.969570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:02.970206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:02.970517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:02.970798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:02.972605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:03.083634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:03.085790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:03.089800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:03.105208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 748/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:17:03.154326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19409b-cc2f-4ded-b05b-89f42f8c58a8 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:03.154785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19409b-cc2f-4ded-b05b-89f42f8c58a8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:03.154897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19409b-cc2f-4ded-b05b-89f42f8c58a8 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:03.156268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 758/1506] 10.222.0.49 () {52 vars in 830 bytes} [Tue Aug 2 10:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:17:03.160222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:03.161462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:03.272892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799fc7b-de33-45ee-9941-8dd777562899 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:03.370576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:03.371143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:03.371457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:03.371747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] PATH_INFO: `/v3/users/1a8e0b1bad8d4162a4791cf1fecfc114` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:03.385046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a8e0b1bad8d4162a4791cf1fecfc114)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:03.386071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:03.386071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: warnings.warn(msg) Aug 02 10:17:03.386071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: Aug 02 10:17:03.387311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:03.396757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:03.647627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a8e0b1bad8d4162a4791cf1fecfc114'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.647989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a8e0b1bad8d4162a4791cf1fecfc114'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.662937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a8e0b1bad8d4162a4791cf1fecfc114'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.762428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a8e0b1bad8d4162a4791cf1fecfc114'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.789959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a8e0b1bad8d4162a4791cf1fecfc114'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:03.795887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 749/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:17:03 2022] DELETE /identity/v3/users/1a8e0b1bad8d4162a4791cf1fecfc114 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:03.801008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:03.801304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:03.801588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:03.859760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:17:03.860993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:03.938735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:03.951164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 759/1508] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:17:03.965117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:03.966395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:04.069480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736e8dd-5cc1-46f5-a00b-61dfa82ee4c8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.125947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:04.145272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42810d-4fb9-48e6-9750-2500f48317f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.149170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42810d-4fb9-48e6-9750-2500f48317f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.149170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42810d-4fb9-48e6-9750-2500f48317f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.149170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42810d-4fb9-48e6-9750-2500f48317f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.149170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3f42810d-4fb9-48e6-9750-2500f48317f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:04.261154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42810d-4fb9-48e6-9750-2500f48317f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.262614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42810d-4fb9-48e6-9750-2500f48317f3 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:04.282979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 750/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:04.841767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:04.841767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:17:04.853236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc08c8-afa5-46b9-b245-e1568825b07f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:04.958891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:04.959767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:04.960050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:04.960382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] PATH_INFO: `/v3/projects/1dcb251d85fa486596bc3dda8a99df4e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:04.977064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dcb251d85fa486596bc3dda8a99df4e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:04.978989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:17:04.978989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:17:04.978989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:17:04.981383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:05.214451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dcb251d85fa486596bc3dda8a99df4e', 'request_id': 'req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:05.385076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dcb251d85fa486596bc3dda8a99df4e', 'request_id': 'req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:05.392084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dcb251d85fa486596bc3dda8a99df4e', 'request_id': 'req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:05.424186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] The token cache is being invalidate because project 1dcb251d85fa486596bc3dda8a99df4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:17:05.424476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:05.428858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7b9c5f5e-535a-4a39-b73f-f25a7b8bcc1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:17:05.428858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 760/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:17:04 2022] DELETE /identity/v3/projects/1dcb251d85fa486596bc3dda8a99df4e => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:17:14.280003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=62 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:15.421684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:01.703988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c793b1b-4f59-47ff-8909-284f95957476 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:01.704613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c793b1b-4f59-47ff-8909-284f95957476 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:01.704720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c793b1b-4f59-47ff-8909-284f95957476 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:01.706133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:01.712080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60dfc7-2920-429b-b1de-2624afbde7ec None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:01.712616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60dfc7-2920-429b-b1de-2624afbde7ec None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:01.712616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60dfc7-2920-429b-b1de-2624afbde7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:01.815495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5e60dfc7-2920-429b-b1de-2624afbde7ec None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:01.815495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5e60dfc7-2920-429b-b1de-2624afbde7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:01.925322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:02.081137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c793b1b-4f59-47ff-8909-284f95957476 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:02.083496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5c793b1b-4f59-47ff-8909-284f95957476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:02.214807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c793b1b-4f59-47ff-8909-284f95957476 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:02.220471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3c94a-1821-4ae1-b5ec-3b578ddbc4e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:02.221160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3c94a-1821-4ae1-b5ec-3b578ddbc4e7 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:02.221451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3c94a-1821-4ae1-b5ec-3b578ddbc4e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:02.221725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3c94a-1821-4ae1-b5ec-3b578ddbc4e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:02.231065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3c94a-1821-4ae1-b5ec-3b578ddbc4e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:02.232703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3c94a-1821-4ae1-b5ec-3b578ddbc4e7 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:02.243912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 752/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:03.141781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab67621-d94e-44b1-9f87-9223dda9771b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:03.142466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab67621-d94e-44b1-9f87-9223dda9771b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:03.142466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab67621-d94e-44b1-9f87-9223dda9771b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:03.144026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:03.153925 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0d26-49fb-404b-abea-5b14792dff00 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:03.154499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0d26-49fb-404b-abea-5b14792dff00 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:03.154930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0d26-49fb-404b-abea-5b14792dff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:03.190076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-41bc0d26-49fb-404b-abea-5b14792dff00 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:03.192187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-41bc0d26-49fb-404b-abea-5b14792dff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:03.204951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:03.368499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab67621-d94e-44b1-9f87-9223dda9771b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:03.384140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab67621-d94e-44b1-9f87-9223dda9771b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:03.389702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c0ba6-93da-489e-9a0e-5c2c5a14bc33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:03.390338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c0ba6-93da-489e-9a0e-5c2c5a14bc33 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:03.390619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c0ba6-93da-489e-9a0e-5c2c5a14bc33 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:03.390895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c0ba6-93da-489e-9a0e-5c2c5a14bc33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:03.401269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c0ba6-93da-489e-9a0e-5c2c5a14bc33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:03.403017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c0ba6-93da-489e-9a0e-5c2c5a14bc33 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:03.414798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 763/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:03.749145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be67525-53aa-4416-9c3b-9c1c2a9a6d4b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:03.749435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be67525-53aa-4416-9c3b-9c1c2a9a6d4b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:03.749690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be67525-53aa-4416-9c3b-9c1c2a9a6d4b None None] PATH_INFO: `/v3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:03.751041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 754/1517] 10.222.0.89 () {54 vars in 925 bytes} [Tue Aug 2 10:19:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:19:03.759570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2dbef-62e0-47d6-a3a0-3b4632450ddb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:03.759853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2dbef-62e0-47d6-a3a0-3b4632450ddb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:03.760131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2dbef-62e0-47d6-a3a0-3b4632450ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:03.832691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-88b2dbef-62e0-47d6-a3a0-3b4632450ddb None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:03.834083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-88b2dbef-62e0-47d6-a3a0-3b4632450ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:03.854576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:03.919637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 764/1518] 10.222.0.89 () {58 vars in 1015 bytes} [Tue Aug 2 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:08.278998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5bd69-2b71-40df-a7b0-b2fd4eaca059 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:08.279774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5bd69-2b71-40df-a7b0-b2fd4eaca059 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:08.279774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5bd69-2b71-40df-a7b0-b2fd4eaca059 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:08.281245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 755/1519] 10.222.0.49 () {54 vars in 961 bytes} [Tue Aug 2 10:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:08.286928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a82cf6-654e-4cc9-8ee8-0cee1200b017 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:08.287461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a82cf6-654e-4cc9-8ee8-0cee1200b017 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:08.287593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a82cf6-654e-4cc9-8ee8-0cee1200b017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:08.309463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-90a82cf6-654e-4cc9-8ee8-0cee1200b017 None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:08.310701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-90a82cf6-654e-4cc9-8ee8-0cee1200b017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:08.320017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 765/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Tue Aug 2 10:19:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:08.327820 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5bd69-2b71-40df-a7b0-b2fd4eaca059 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:08.343300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5bd69-2b71-40df-a7b0-b2fd4eaca059 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:08.348398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef238f-9c6e-4000-bb40-5baa4f5c074d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:08.349019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef238f-9c6e-4000-bb40-5baa4f5c074d service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:08.349303 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef238f-9c6e-4000-bb40-5baa4f5c074d service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:08.349583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef238f-9c6e-4000-bb40-5baa4f5c074d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:08.359513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef238f-9c6e-4000-bb40-5baa4f5c074d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:08.361139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef238f-9c6e-4000-bb40-5baa4f5c074d service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:08.371921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 756/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:19:14.718179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018cfe-ba96-491f-87e4-6ea5db441d9a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:14.718854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018cfe-ba96-491f-87e4-6ea5db441d9a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:14.718854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018cfe-ba96-491f-87e4-6ea5db441d9a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:14.720174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:14.726156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a17d6-968e-48ed-aed6-27161011a9b0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:14.726708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a17d6-968e-48ed-aed6-27161011a9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:14.726708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a17d6-968e-48ed-aed6-27161011a9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:14.748290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-331a17d6-968e-48ed-aed6-27161011a9b0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:14.749526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-331a17d6-968e-48ed-aed6-27161011a9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:14.757851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:14.920306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73018cfe-ba96-491f-87e4-6ea5db441d9a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:14.936092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73018cfe-ba96-491f-87e4-6ea5db441d9a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:14.941857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7715b-1057-46a5-95e5-e467b626434e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:14.942554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7715b-1057-46a5-95e5-e467b626434e service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:14.942894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7715b-1057-46a5-95e5-e467b626434e service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:14.943173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7715b-1057-46a5-95e5-e467b626434e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:14.952677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7715b-1057-46a5-95e5-e467b626434e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:14.954455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7715b-1057-46a5-95e5-e467b626434e service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:14.964994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 767/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:15.812893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab89e60-100b-4554-b031-4700167b330b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:15.813479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab89e60-100b-4554-b031-4700167b330b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:15.813785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab89e60-100b-4554-b031-4700167b330b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:15.816434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:19:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:15.822717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6e9a-9f33-4e3e-a1aa-8af04258621c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:15.823539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6e9a-9f33-4e3e-a1aa-8af04258621c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:15.823539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6e9a-9f33-4e3e-a1aa-8af04258621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:15.846107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cd6a6e9a-9f33-4e3e-a1aa-8af04258621c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:15.847446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cd6a6e9a-9f33-4e3e-a1aa-8af04258621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:15.855992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:16.023056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab89e60-100b-4554-b031-4700167b330b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:16.041223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab89e60-100b-4554-b031-4700167b330b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:16.047311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd81cc8-5e1d-40ec-9a80-600a99348943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:16.047963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd81cc8-5e1d-40ec-9a80-600a99348943 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:16.048245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd81cc8-5e1d-40ec-9a80-600a99348943 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:16.048522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd81cc8-5e1d-40ec-9a80-600a99348943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:16.063120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd81cc8-5e1d-40ec-9a80-600a99348943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:16.066049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd81cc8-5e1d-40ec-9a80-600a99348943 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:16.084433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 759/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:20.409519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c05331-dd4a-4f1e-8d68-63dde246a8b9 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:20.410176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c05331-dd4a-4f1e-8d68-63dde246a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:20.410176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c05331-dd4a-4f1e-8d68-63dde246a8b9 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:20.411522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:19:20.417814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396e7a7-a445-4e13-94c2-6705ae996375 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:20.418352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396e7a7-a445-4e13-94c2-6705ae996375 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:20.418352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396e7a7-a445-4e13-94c2-6705ae996375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:20.438194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9396e7a7-a445-4e13-94c2-6705ae996375 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:20.439456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9396e7a7-a445-4e13-94c2-6705ae996375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:20.447693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:20.530338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c05331-dd4a-4f1e-8d68-63dde246a8b9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:20.533267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-68c05331-dd4a-4f1e-8d68-63dde246a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:20.629001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c05331-dd4a-4f1e-8d68-63dde246a8b9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:20.633011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233cd8d-9bc4-4088-b5c8-43f05d3ed2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:20.633644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233cd8d-9bc4-4088-b5c8-43f05d3ed2ec service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:20.633929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233cd8d-9bc4-4088-b5c8-43f05d3ed2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:20.634205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233cd8d-9bc4-4088-b5c8-43f05d3ed2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:20.645004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233cd8d-9bc4-4088-b5c8-43f05d3ed2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:20.646537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233cd8d-9bc4-4088-b5c8-43f05d3ed2ec service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:20.661313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 770/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:22.748870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc25aea-e1d6-4a47-8f55-09f63ab5b270 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:22.749535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc25aea-e1d6-4a47-8f55-09f63ab5b270 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:22.749535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc25aea-e1d6-4a47-8f55-09f63ab5b270 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:22.750851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:22.757932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbef17-7ea1-4aa7-b3f2-e5b241ce15a9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:22.758779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbef17-7ea1-4aa7-b3f2-e5b241ce15a9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:22.758779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbef17-7ea1-4aa7-b3f2-e5b241ce15a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:22.779869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-59bbef17-7ea1-4aa7-b3f2-e5b241ce15a9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:22.781195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-59bbef17-7ea1-4aa7-b3f2-e5b241ce15a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:22.790049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:19:22.828358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc25aea-e1d6-4a47-8f55-09f63ab5b270 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:22.839972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc25aea-e1d6-4a47-8f55-09f63ab5b270 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:22.843752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef37a30-c9ee-4344-9769-3297e39c9738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:22.844392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef37a30-c9ee-4344-9769-3297e39c9738 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:22.844674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef37a30-c9ee-4344-9769-3297e39c9738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:22.844948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef37a30-c9ee-4344-9769-3297e39c9738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:22.855099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef37a30-c9ee-4344-9769-3297e39c9738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:22.856626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef37a30-c9ee-4344-9769-3297e39c9738 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:22.868453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 762/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:19:26.665380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c6646-9d69-4a3f-8ca3-77910ac43b99 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:26.666279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c6646-9d69-4a3f-8ca3-77910ac43b99 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:26.666459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c6646-9d69-4a3f-8ca3-77910ac43b99 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:26.667980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:19:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:26.674222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037704d-6bbf-4e14-b1c8-8cf011b79b61 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:26.675116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037704d-6bbf-4e14-b1c8-8cf011b79b61 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:26.675116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037704d-6bbf-4e14-b1c8-8cf011b79b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:26.682980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:26.696156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1037704d-6bbf-4e14-b1c8-8cf011b79b61 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:26.697406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1037704d-6bbf-4e14-b1c8-8cf011b79b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:26.705599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:27.022829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c6646-9d69-4a3f-8ca3-77910ac43b99 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:27.024826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-705c6646-9d69-4a3f-8ca3-77910ac43b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:27.105466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:27.121693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c6646-9d69-4a3f-8ca3-77910ac43b99 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:27.125514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757784c-2deb-452e-b52e-a8defa3a2e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:27.126163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757784c-2deb-452e-b52e-a8defa3a2e8e service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:27.126480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757784c-2deb-452e-b52e-a8defa3a2e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:27.126987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757784c-2deb-452e-b52e-a8defa3a2e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:27.137037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757784c-2deb-452e-b52e-a8defa3a2e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:27.138559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757784c-2deb-452e-b52e-a8defa3a2e8e service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:27.150590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 773/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:29.258168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648eb48-9448-4567-8793-5cc22747dee1 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:29.258755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648eb48-9448-4567-8793-5cc22747dee1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:29.258755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648eb48-9448-4567-8793-5cc22747dee1 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:29.260241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:19:29.266124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721535e-d45e-4f43-8433-5b156bc911ba None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:29.266645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721535e-d45e-4f43-8433-5b156bc911ba None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:29.266746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721535e-d45e-4f43-8433-5b156bc911ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:29.288267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d721535e-d45e-4f43-8433-5b156bc911ba None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:19:29.289511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d721535e-d45e-4f43-8433-5b156bc911ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:29.297872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:19:29.612532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648eb48-9448-4567-8793-5cc22747dee1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:29.614247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3648eb48-9448-4567-8793-5cc22747dee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:19:29.629562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648eb48-9448-4567-8793-5cc22747dee1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:29.633986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234357ed-c28d-4c36-8eb4-bf36c956c910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:29.634633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234357ed-c28d-4c36-8eb4-bf36c956c910 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:29.634917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234357ed-c28d-4c36-8eb4-bf36c956c910 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:29.635231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234357ed-c28d-4c36-8eb4-bf36c956c910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:29.646058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234357ed-c28d-4c36-8eb4-bf36c956c910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:29.647613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234357ed-c28d-4c36-8eb4-bf36c956c910 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:29.658647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 765/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:39.282387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=11 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:39.653755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=8 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:02.466562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8cd3f-df9b-4620-a974-3786b5a82b75 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.467480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8cd3f-df9b-4620-a974-3786b5a82b75 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.467480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8cd3f-df9b-4620-a974-3786b5a82b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.492729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ddd8cd3f-df9b-4620-a974-3786b5a82b75 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:02.494032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ddd8cd3f-df9b-4620-a974-3786b5a82b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:02.506504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 775/1540] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:02.515912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1e3ac-2bde-4359-9383-6c199626843a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.518504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1e3ac-2bde-4359-9383-6c199626843a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.518504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1e3ac-2bde-4359-9383-6c199626843a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.557034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-95e1e3ac-2bde-4359-9383-6c199626843a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:02.561578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-95e1e3ac-2bde-4359-9383-6c199626843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:02.571474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 766/1541] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:02.576522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8cd3f-df9b-4620-a974-3786b5a82b75 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:02.592946 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8cd3f-df9b-4620-a974-3786b5a82b75 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:02.604267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87ba6d-1905-4b40-a8b6-ebf87cc36305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:02.604267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87ba6d-1905-4b40-a8b6-ebf87cc36305 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.604267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87ba6d-1905-4b40-a8b6-ebf87cc36305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.604267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87ba6d-1905-4b40-a8b6-ebf87cc36305 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.604655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87ba6d-1905-4b40-a8b6-ebf87cc36305 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:02.604655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87ba6d-1905-4b40-a8b6-ebf87cc36305 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:02.604655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87ba6d-1905-4b40-a8b6-ebf87cc36305 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:02.628019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 776/1542] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:02.640035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1e3ac-2bde-4359-9383-6c199626843a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:02.652834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1e3ac-2bde-4359-9383-6c199626843a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:02.661194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf8291-7458-4134-a045-4baf3c0678c0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.667364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf8291-7458-4134-a045-4baf3c0678c0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.667364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf8291-7458-4134-a045-4baf3c0678c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.667929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1939648-3ed5-43b8-b424-92c628c2232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:02.668612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939648-3ed5-43b8-b424-92c628c2232b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.668899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939648-3ed5-43b8-b424-92c628c2232b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.669183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939648-3ed5-43b8-b424-92c628c2232b admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.670043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1939648-3ed5-43b8-b424-92c628c2232b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:02.672097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1939648-3ed5-43b8-b424-92c628c2232b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:02.695330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b2cf8291-7458-4134-a045-4baf3c0678c0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:02.701577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b2cf8291-7458-4134-a045-4baf3c0678c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:02.707735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 777/1543] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:02.717802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e056e6a-2fce-49d1-8179-2031eb7d72e2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.718009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e056e6a-2fce-49d1-8179-2031eb7d72e2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.718284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e056e6a-2fce-49d1-8179-2031eb7d72e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.753722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5e056e6a-2fce-49d1-8179-2031eb7d72e2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:02.754959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5e056e6a-2fce-49d1-8179-2031eb7d72e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:02.765495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 778/1544] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:02.770722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e056e6a-2fce-49d1-8179-2031eb7d72e2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:02.785461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e056e6a-2fce-49d1-8179-2031eb7d72e2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:02.793812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c38495-245d-4b6f-9f4f-8eb0e5fe94d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:02.794461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c38495-245d-4b6f-9f4f-8eb0e5fe94d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.794750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c38495-245d-4b6f-9f4f-8eb0e5fe94d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.795032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c38495-245d-4b6f-9f4f-8eb0e5fe94d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.795773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c38495-245d-4b6f-9f4f-8eb0e5fe94d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:02.796090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c38495-245d-4b6f-9f4f-8eb0e5fe94d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:02.797755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c38495-245d-4b6f-9f4f-8eb0e5fe94d0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:02.814222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 779/1545] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:21:02.815603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a45f-4ae2-4209-934e-ed569a151990 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.815905 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a45f-4ae2-4209-934e-ed569a151990 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.816191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a45f-4ae2-4209-934e-ed569a151990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.850174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6230a45f-4ae2-4209-934e-ed569a151990 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:02.851526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6230a45f-4ae2-4209-934e-ed569a151990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:02.855039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 767/1546] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:02.856491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f272a-3bcb-41a3-b0f1-5fc9311a2857 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.856783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f272a-3bcb-41a3-b0f1-5fc9311a2857 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.857064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f272a-3bcb-41a3-b0f1-5fc9311a2857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.903881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-de0f272a-3bcb-41a3-b0f1-5fc9311a2857 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:02.905102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-de0f272a-3bcb-41a3-b0f1-5fc9311a2857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:02.934472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 780/1547] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:02.935086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230a45f-4ae2-4209-934e-ed569a151990 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:02.951300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230a45f-4ae2-4209-934e-ed569a151990 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:02.956098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e21d2-894b-4fd9-8fdd-d565967033ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:02.957088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e21d2-894b-4fd9-8fdd-d565967033ce admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.957738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e21d2-894b-4fd9-8fdd-d565967033ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.958023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e21d2-894b-4fd9-8fdd-d565967033ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.958705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e21d2-894b-4fd9-8fdd-d565967033ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:02.959807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-0a8e21d2-894b-4fd9-8fdd-d565967033ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:21:02.959807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:21:02.959807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:21:02.961060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e21d2-894b-4fd9-8fdd-d565967033ce admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:02.976543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 768/1548] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:02.977315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f272a-3bcb-41a3-b0f1-5fc9311a2857 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:02.988894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f272a-3bcb-41a3-b0f1-5fc9311a2857 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:02.993807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8807f-7602-4795-96a4-dba4368674d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:02.994428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8807f-7602-4795-96a4-dba4368674d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:02.994719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8807f-7602-4795-96a4-dba4368674d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:02.994999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8807f-7602-4795-96a4-dba4368674d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:02.995730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8807f-7602-4795-96a4-dba4368674d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:02.996769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-7ed8807f-7602-4795-96a4-dba4368674d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:21:02.996769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: warnings.warn(msg) Aug 02 10:21:02.996769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: Aug 02 10:21:02.997990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8807f-7602-4795-96a4-dba4368674d4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.045018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 781/1549] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:03.045018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cf876-1db0-40e7-af92-d3b084605365 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.045018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cf876-1db0-40e7-af92-d3b084605365 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.045018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cf876-1db0-40e7-af92-d3b084605365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.063827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0a9cf876-1db0-40e7-af92-d3b084605365 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.065082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0a9cf876-1db0-40e7-af92-d3b084605365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.074363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 769/1550] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:02 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:03.075907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25020663-3758-4054-af3f-c0193c1288d6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.078803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25020663-3758-4054-af3f-c0193c1288d6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.078803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25020663-3758-4054-af3f-c0193c1288d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.097974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-25020663-3758-4054-af3f-c0193c1288d6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.099218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-25020663-3758-4054-af3f-c0193c1288d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.130246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 782/1551] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.131781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75c2f-6ceb-4727-9b68-34e51624850d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.132070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75c2f-6ceb-4727-9b68-34e51624850d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.132348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75c2f-6ceb-4727-9b68-34e51624850d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.158357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b1c75c2f-6ceb-4727-9b68-34e51624850d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.160278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b1c75c2f-6ceb-4727-9b68-34e51624850d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.168913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 783/1552] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.170269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adf27f-817f-4f24-bb98-35be06f6d65f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.170562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adf27f-817f-4f24-bb98-35be06f6d65f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.170847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adf27f-817f-4f24-bb98-35be06f6d65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.181776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 770/1553] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.183220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696714a-f299-40a0-816f-9cc148e2fcce None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.183612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696714a-f299-40a0-816f-9cc148e2fcce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.183983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696714a-f299-40a0-816f-9cc148e2fcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.193042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b5adf27f-817f-4f24-bb98-35be06f6d65f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.194299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b5adf27f-817f-4f24-bb98-35be06f6d65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.202734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 784/1554] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.204057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64a4e0-b587-4b24-be8e-d211a250306e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.204352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64a4e0-b587-4b24-be8e-d211a250306e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.204910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64a4e0-b587-4b24-be8e-d211a250306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.208384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a696714a-f299-40a0-816f-9cc148e2fcce None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.209666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a696714a-f299-40a0-816f-9cc148e2fcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.217909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 771/1555] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.218839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696714a-f299-40a0-816f-9cc148e2fcce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.226844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4b64a4e0-b587-4b24-be8e-d211a250306e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.228022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4b64a4e0-b587-4b24-be8e-d211a250306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.230520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696714a-f299-40a0-816f-9cc148e2fcce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.235457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.236086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.236369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.236648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.237037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 785/1556] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.237319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.238088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64a4e0-b587-4b24-be8e-d211a250306e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.239485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.249176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64a4e0-b587-4b24-be8e-d211a250306e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.254011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01c044-0d42-42cf-8209-8237de65f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.254702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01c044-0d42-42cf-8209-8237de65f25f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.254993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01c044-0d42-42cf-8209-8237de65f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.255284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01c044-0d42-42cf-8209-8237de65f25f admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.255979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01c044-0d42-42cf-8209-8237de65f25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.257597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01c044-0d42-42cf-8209-8237de65f25f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.268300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 786/1557] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:03.270036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269627e-1021-4b81-8d77-c508298b84e9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.270331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269627e-1021-4b81-8d77-c508298b84e9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.270614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269627e-1021-4b81-8d77-c508298b84e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.278258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:03.291379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d269627e-1021-4b81-8d77-c508298b84e9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.292612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d269627e-1021-4b81-8d77-c508298b84e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.301098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 787/1558] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.302406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed335b0-0d14-49c3-b148-9d9e3376c21e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.302695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed335b0-0d14-49c3-b148-9d9e3376c21e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.302970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed335b0-0d14-49c3-b148-9d9e3376c21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.322415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9ed335b0-0d14-49c3-b148-9d9e3376c21e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.323675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9ed335b0-0d14-49c3-b148-9d9e3376c21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.332171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 788/1559] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.333361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236d3b5-8031-4028-bbea-2b45093acbfc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.333646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236d3b5-8031-4028-bbea-2b45093acbfc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.333920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236d3b5-8031-4028-bbea-2b45093acbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.341856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 772/1560] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:03.343056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.354141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a236d3b5-8031-4028-bbea-2b45093acbfc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.355087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85149f86-365b-4f91-b699-a1c6495dbd58 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.355402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a236d3b5-8031-4028-bbea-2b45093acbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.360354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.361256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.361667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.361948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.362670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:03.363009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.364096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:21:03.364096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: warnings.warn(msg) Aug 02 10:21:03.364096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: Aug 02 10:21:03.365001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.369426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 789/1561] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.369426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19310-1be7-490e-af7b-d1aabbc9bc04 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.369426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19310-1be7-490e-af7b-d1aabbc9bc04 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.369426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19310-1be7-490e-af7b-d1aabbc9bc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.378413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 773/1562] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:03.379379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.389819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4eb97-d5bd-4051-a0f4-38d4666716f1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.394839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2bb19310-1be7-490e-af7b-d1aabbc9bc04 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:03.395872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.396108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2bb19310-1be7-490e-af7b-d1aabbc9bc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:03.396512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.396804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.397078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.397784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:03.398091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.399874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.402593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:03.406915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 790/1563] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:03.408399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb19310-1be7-490e-af7b-d1aabbc9bc04 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.414079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 774/1564] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:03.415158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.421079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb19310-1be7-490e-af7b-d1aabbc9bc04 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.425925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.426562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.426812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1dd0a-eb4d-43f9-8458-875f81ffe584 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.426934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.427250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.428021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.429624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.431744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.432362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.432646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.432995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.433710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:03.434013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.435635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.441364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 791/1565] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:03.442100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.452410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594beca6-5c63-472d-bbe0-c905cd7cfa17 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.453006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 775/1566] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:03.454397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.458072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.458770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.459123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.459499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.460269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:03.460660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.462399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.470391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8106dee-d9c6-4709-9de2-8eeb04ccf1e8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.475746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.476365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.476648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.476984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.477625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.477943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 792/1567] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:03.478849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.479297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.490560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 776/1568] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:03.491412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.495541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567a0f-a581-48c9-ae4f-bdf712744a19 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.501046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.501204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e31776-0fc2-4b4c-8736-4647bf975ff3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.501820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.502600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.502600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.503624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:03.503939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.505539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.506278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.506899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.507208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.507498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.508165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.510076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.522038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 793/1569] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:03.523368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.537154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43041dae-77e1-4556-899c-ceb3d271ddff admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.540246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.540920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.541208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.541492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.542193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:03.542513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.544171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.557142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 794/1570] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:03.558068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.569428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1d35b-37a3-41d1-a2be-6d89c165aaa1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.574007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.574694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.574958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.575267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.575945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.577858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.636354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 777/1571] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:03.637269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.658594 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 795/1572] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:03.662694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.707773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5097c-5214-4899-9fcb-fc51b2304269 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.731444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df31215-3901-46cb-b63c-e499091dcf3e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.736453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.737080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.737372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.737687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.738359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.740387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.766252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.771672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.771672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.771672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] PATH_INFO: `/v3/projects/0dc1c45a26e84a3bbc06d52fec42e332/users/d2ee2b6dd6f647179d2af999b3e62788/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.785204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc1c45a26e84a3bbc06d52fec42e332, user_id=d2ee2b6dd6f647179d2af999b3e62788, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.788797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.853541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 796/1573] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:03.854438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.869792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 778/1574] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:03 2022] PUT /identity/v3/projects/0dc1c45a26e84a3bbc06d52fec42e332/users/d2ee2b6dd6f647179d2af999b3e62788/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:03.874792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.913230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:03.918622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:03.928766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae648d2-5fd5-4eb6-8c91-d3fe0275ff15 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.941524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991223a1-c6c3-4a35-8890-5839fd384123 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.946811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.947501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.947789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.948103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.948840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.950468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.964532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 779/1575] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:03.964532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:03.973209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c75c-1c2f-4234-86ba-fccbc7107a82 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:03.980048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.980784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.981075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.981408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.983819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.984234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:03.994533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:03.995317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:03.995674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:03.996015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:03.996697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:03.998713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.061115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 780/1576] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.062592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.093383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 797/1577] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:03 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.094265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.146809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23152d-e052-487e-bf29-83c7cd68953a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.168598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cbb34-4881-4d8c-969d-8c786b11fabc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.184214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.184882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.185232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.185521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.186208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.188358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.224178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.230057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.230057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.230057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.230057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.230057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.296959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 798/1578] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.335442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.335442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4c964-7a34-4d73-a92a-a012af930bc5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.335442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85066f-590c-464f-9a22-c4cc9cd4c82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.335442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85066f-590c-464f-9a22-c4cc9cd4c82c admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.335867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85066f-590c-464f-9a22-c4cc9cd4c82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.335867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85066f-590c-464f-9a22-c4cc9cd4c82c admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.335867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85066f-590c-464f-9a22-c4cc9cd4c82c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.335867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85066f-590c-464f-9a22-c4cc9cd4c82c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.441599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 781/1579] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/projects => generated 379 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.443102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.490730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:04.504472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dccfed-1178-41e0-bd23-368646be3e84 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.509273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.512058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.512058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.512058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.512058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.513077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.515047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 799/1580] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.516512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24707e-ecaa-46d4-ac46-867262849933 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.516804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24707e-ecaa-46d4-ac46-867262849933 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.517086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24707e-ecaa-46d4-ac46-867262849933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.537395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5f24707e-ecaa-46d4-ac46-867262849933 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:04.540546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5f24707e-ecaa-46d4-ac46-867262849933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:04.546349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 800/1581] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:04.547156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24707e-ecaa-46d4-ac46-867262849933 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.557571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24707e-ecaa-46d4-ac46-867262849933 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.562207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.563685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.563685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.563685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] PATH_INFO: `/v3/projects/5db81d6f81114ce6a527b6e718453fc5/users/04456e2f28b14ff28857b7d22b4a468e/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.578886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db81d6f81114ce6a527b6e718453fc5, user_id=04456e2f28b14ff28857b7d22b4a468e, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.579947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:21:04.579947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:21:04.579947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:21:04.582441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.669932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 782/1582] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.669932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.678800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d90cbf-eaf6-4f39-af67-bb8d5b97c9ff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.683465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.684078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.684360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.684637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.685299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.687316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.697977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 801/1583] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:04 2022] PUT /identity/v3/projects/5db81d6f81114ce6a527b6e718453fc5/users/04456e2f28b14ff28857b7d22b4a468e/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:04.698921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.729162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:04.765964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbaba-48c5-4064-947b-423631604516 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.771585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.772218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.772536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.772819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.773494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.775627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.820170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 783/1584] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.820170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.830843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05921d1b-b5c9-47b6-8c5a-df5ae1f5b388 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.836316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.836929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.837219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.837529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.838370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.839955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.851046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 784/1585] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:04.852048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.862715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8b2b-e7c5-456e-8ee1-cf655f8a7deb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.867731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.868384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.868647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.868929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.869598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.871653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.876326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 802/1586] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.876326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.887182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa38ca-bf6a-4b75-b9c7-f1927236867b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.892021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.892643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.892925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.893201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.894394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.896023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.906364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 803/1587] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:04.911540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.965200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d535bb7-3177-48f4-89bf-cf533832c051 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.968049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 785/1588] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:04.968049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:04.972170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d967f70-7bf3-467b-aedd-c55824803844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.973099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d967f70-7bf3-467b-aedd-c55824803844 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.974673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d967f70-7bf3-467b-aedd-c55824803844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.975116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d967f70-7bf3-467b-aedd-c55824803844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.978230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befab65-9791-45cd-8f4b-55555c7e028d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:04.982806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:04.983444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:04.983924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:04.984199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:04.985157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.985652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d967f70-7bf3-467b-aedd-c55824803844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:04.986896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.987180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d967f70-7bf3-467b-aedd-c55824803844 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:04.997370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 804/1589] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:05.004672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 786/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.004672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.004993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0e0a8-dbf7-4c6d-93fd-31f2151e6636 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.004993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0e0a8-dbf7-4c6d-93fd-31f2151e6636 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.004993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0e0a8-dbf7-4c6d-93fd-31f2151e6636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.012022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc8521-8471-46ba-a56d-2fc3a035f013 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.017627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.018028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.018356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.018599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.019903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.021500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.030659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cee0e0a8-dbf7-4c6d-93fd-31f2151e6636 None None] MFA Rules not processed for user `04456e2f28b14ff28857b7d22b4a468e`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:05.031936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cee0e0a8-dbf7-4c6d-93fd-31f2151e6636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:05.032790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 787/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.033635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.043571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2391837-0e3e-4e57-86a1-5edbc96bb1c0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.048453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.049070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.049351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.049621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.050386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.052015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.062374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 788/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.063249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.073870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f9747-3546-444d-a767-17e697600784 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.078422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.079036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.079352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.079632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.080326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.082550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.094725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 789/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.095837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.108188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd42833-9a23-4ba9-a491-a4319d358b83 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.111180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 805/1594] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:05.111512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0e0a8-dbf7-4c6d-93fd-31f2151e6636 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.123632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.124052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.124643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.125083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.126123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.127955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.128769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0e0a8-dbf7-4c6d-93fd-31f2151e6636 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.134927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.135658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.136920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.137184 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.138501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.140774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.147586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 790/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.147586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.161085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 806/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.161085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.164319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa51eb-3905-440f-bf63-c419da64739d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.171367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.171883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c274-613f-480d-a6fe-28aad7d84173 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.172279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.172675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.172950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.173846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.176065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.180051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.180051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.180051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.180051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.180405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.180405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.190032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 791/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.191574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.194640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 807/1598] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.195600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.203646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942dc3b-f33b-4eca-84c5-0cbe7da54038 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.205593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d863c8-2471-4f72-b911-d051a4eb8d47 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.211990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.212726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.213019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.213302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.213564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532b9b5-3821-46f4-8611-28fc3b2cdb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.214021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.214237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532b9b5-3821-46f4-8611-28fc3b2cdb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.214518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532b9b5-3821-46f4-8611-28fc3b2cdb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.214792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532b9b5-3821-46f4-8611-28fc3b2cdb85 admin admin] PATH_INFO: `/v3/projects/855d0942718247168191f1760fa678b2/users/196de827124f47c887637b59e7e346ea/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.215831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.221927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:05.229188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 808/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.229826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.233509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532b9b5-3821-46f4-8611-28fc3b2cdb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=855d0942718247168191f1760fa678b2, user_id=196de827124f47c887637b59e7e346ea, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.236545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532b9b5-3821-46f4-8611-28fc3b2cdb85 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.242244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370acbc-87bb-4193-b3e6-5b9c0f80188d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.246864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.247562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.247850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.248129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] PATH_INFO: `/v3/projects/d43f7e6aca9a47c38799a19001c387f4/users/968296b8db55464a91aedbd1dfecc1b3/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.264826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d43f7e6aca9a47c38799a19001c387f4, user_id=968296b8db55464a91aedbd1dfecc1b3, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.267783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.335649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 792/1600] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:05 2022] PUT /identity/v3/projects/855d0942718247168191f1760fa678b2/users/196de827124f47c887637b59e7e346ea/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:05.335649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e1f7-d30a-4de7-a124-8783a3903429 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.335649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e1f7-d30a-4de7-a124-8783a3903429 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.335649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e1f7-d30a-4de7-a124-8783a3903429 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.335649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 793/1601] 10.222.0.49 () {54 vars in 962 bytes} [Tue Aug 2 10:21:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:05.335649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78e1f7-d30a-4de7-a124-8783a3903429 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.350763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 809/1602] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:05 2022] PUT /identity/v3/projects/d43f7e6aca9a47c38799a19001c387f4/users/968296b8db55464a91aedbd1dfecc1b3/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:05.351939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.423970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78e1f7-d30a-4de7-a124-8783a3903429 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.429266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.429924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.430219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.430505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.430978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875c5c7-5ad4-4410-a153-299c2b66b922 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.431384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.433130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.434565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:05.435976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.436388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.436669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.436945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] PATH_INFO: `/v3/projects/8da396d25bf44b8d89ab2f28f0953db4/users/998e04c59ad64d3e98517ee84c4268a1/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.448245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 794/1603] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:05.448245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.454490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da396d25bf44b8d89ab2f28f0953db4, user_id=998e04c59ad64d3e98517ee84c4268a1, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.456229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c093ce-803b-4ccb-bc3d-ebec4a95cb30 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.457264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.461168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35e60-3b06-47af-bd60-00653ee55f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.461840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35e60-3b06-47af-bd60-00653ee55f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.462126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35e60-3b06-47af-bd60-00653ee55f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.462406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35e60-3b06-47af-bd60-00653ee55f38 admin admin] PATH_INFO: `/v3/projects/662f508ce05e4a85896f30f772be6410/users/f6880f2f7ad749beb68cd6445f3cc6d6/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.478003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a35e60-3b06-47af-bd60-00653ee55f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=662f508ce05e4a85896f30f772be6410, user_id=f6880f2f7ad749beb68cd6445f3cc6d6, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.480479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a35e60-3b06-47af-bd60-00653ee55f38 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.559908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 810/1604] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:05 2022] PUT /identity/v3/projects/8da396d25bf44b8d89ab2f28f0953db4/users/998e04c59ad64d3e98517ee84c4268a1/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:05.560735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.597407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 795/1605] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:05 2022] PUT /identity/v3/projects/662f508ce05e4a85896f30f772be6410/users/f6880f2f7ad749beb68cd6445f3cc6d6/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:05.599622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774b04-316a-41f5-b16c-d04546b6c302 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.601199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774b04-316a-41f5-b16c-d04546b6c302 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.601746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774b04-316a-41f5-b16c-d04546b6c302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.631847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffba7b3-97ed-4ef8-a9a8-6df694f85264 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.651950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ef774b04-316a-41f5-b16c-d04546b6c302 None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:05.653371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ef774b04-316a-41f5-b16c-d04546b6c302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:05.700349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f7b3e-23fb-4bae-99ca-1494eb0ab9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.701082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f7b3e-23fb-4bae-99ca-1494eb0ab9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.701369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f7b3e-23fb-4bae-99ca-1494eb0ab9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.701649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f7b3e-23fb-4bae-99ca-1494eb0ab9fa admin admin] PATH_INFO: `/v3/projects/74bbbe86d44348c0b1ed8ff50f493e28/users/306ea8d439d94da18d43d91ce9838e19/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.714490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:05.718262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f7b3e-23fb-4bae-99ca-1494eb0ab9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bbbe86d44348c0b1ed8ff50f493e28, user_id=306ea8d439d94da18d43d91ce9838e19, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.721123 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f7b3e-23fb-4bae-99ca-1494eb0ab9fa admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.732663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 796/1606] 10.222.0.49 () {58 vars in 1059 bytes} [Tue Aug 2 10:21:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:05.734019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89549-c1da-4f55-867a-5e8fcd7ab11a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.734382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89549-c1da-4f55-867a-5e8fcd7ab11a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.734977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89549-c1da-4f55-867a-5e8fcd7ab11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.764779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dff89549-c1da-4f55-867a-5e8fcd7ab11a None None] MFA Rules not processed for user `196de827124f47c887637b59e7e346ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:05.767017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dff89549-c1da-4f55-867a-5e8fcd7ab11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:05.824848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 811/1607] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:05 2022] PUT /identity/v3/projects/74bbbe86d44348c0b1ed8ff50f493e28/users/306ea8d439d94da18d43d91ce9838e19/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:05.826364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee17a7f-92a9-42da-8f90-6189ad751544 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.826647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee17a7f-92a9-42da-8f90-6189ad751544 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.826985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee17a7f-92a9-42da-8f90-6189ad751544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.842985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 797/1608] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:05.844369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89549-c1da-4f55-867a-5e8fcd7ab11a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:05.858038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89549-c1da-4f55-867a-5e8fcd7ab11a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:05.860966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3ee17a7f-92a9-42da-8f90-6189ad751544 None None] MFA Rules not processed for user `968296b8db55464a91aedbd1dfecc1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:05.862279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3ee17a7f-92a9-42da-8f90-6189ad751544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:05.903684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:05.928162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:05.928950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.929319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.929669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] PATH_INFO: `/v3/projects/c217a6065f9e4a77a573a915c372b4ea/users/8cbff7e82e0443148d2574deb75ac384/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.937652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 812/1609] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:05.939136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d151b0-8b83-4f6a-9e06-83ccb3c85a16 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:05.939462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d151b0-8b83-4f6a-9e06-83ccb3c85a16 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:05.939741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d151b0-8b83-4f6a-9e06-83ccb3c85a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:05.945137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c217a6065f9e4a77a573a915c372b4ea, user_id=8cbff7e82e0443148d2574deb75ac384, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:05.948014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:05.970243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f5d151b0-8b83-4f6a-9e06-83ccb3c85a16 None None] MFA Rules not processed for user `998e04c59ad64d3e98517ee84c4268a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:05.971649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f5d151b0-8b83-4f6a-9e06-83ccb3c85a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:06.040383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 813/1610] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:06.046306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fe6ab-4af9-4611-b582-16451e5b849b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.046610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fe6ab-4af9-4611-b582-16451e5b849b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.046879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fe6ab-4af9-4611-b582-16451e5b849b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.062711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 798/1611] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:05 2022] PUT /identity/v3/projects/c217a6065f9e4a77a573a915c372b4ea/users/8cbff7e82e0443148d2574deb75ac384/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:06.063004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.071725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5c2fe6ab-4af9-4611-b582-16451e5b849b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:06.073183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5c2fe6ab-4af9-4611-b582-16451e5b849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:06.144832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 814/1612] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:06.146131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857eed6-2227-446e-a18d-0186ad062e03 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.146425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857eed6-2227-446e-a18d-0186ad062e03 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.146703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857eed6-2227-446e-a18d-0186ad062e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.169737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6857eed6-2227-446e-a18d-0186ad062e03 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:06.170203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e512388-c759-46a4-ae71-cd38593c1fa7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.170976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6857eed6-2227-446e-a18d-0186ad062e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:06.176000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.176638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.176961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.177238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.178872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 815/1613] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:06.183147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857eed6-2227-446e-a18d-0186ad062e03 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.187598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:06.244190 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.245975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.246158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857eed6-2227-446e-a18d-0186ad062e03 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.250096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb3dc7-2edf-4deb-9b1d-d6face043dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.250874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb3dc7-2edf-4deb-9b1d-d6face043dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.251015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb3dc7-2edf-4deb-9b1d-d6face043dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.251300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb3dc7-2edf-4deb-9b1d-d6face043dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.255406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 799/1614] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:06.263468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.274855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8874-73e9-4851-9a7b-f684c3cfa750 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.279941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b281f43-805b-4b15-a5e6-39e1f17ccbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.283153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b281f43-805b-4b15-a5e6-39e1f17ccbdb admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.283153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b281f43-805b-4b15-a5e6-39e1f17ccbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.283153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b281f43-805b-4b15-a5e6-39e1f17ccbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.283153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b281f43-805b-4b15-a5e6-39e1f17ccbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.283624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b281f43-805b-4b15-a5e6-39e1f17ccbdb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.322629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb3dc7-2edf-4deb-9b1d-d6face043dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.325127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb3dc7-2edf-4deb-9b1d-d6face043dd9 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.339267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 816/1615] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:06.340850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be086b96-ec78-4748-a255-bcacd5df65ca None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.346360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be086b96-ec78-4748-a255-bcacd5df65ca None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.346360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be086b96-ec78-4748-a255-bcacd5df65ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.385592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-be086b96-ec78-4748-a255-bcacd5df65ca None None] MFA Rules not processed for user `8cbff7e82e0443148d2574deb75ac384`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:06.386958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-be086b96-ec78-4748-a255-bcacd5df65ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:06.410341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 800/1616] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:06.411991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26db97-19db-42c0-b556-ddae36927edf None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.412806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26db97-19db-42c0-b556-ddae36927edf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.413097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26db97-19db-42c0-b556-ddae36927edf None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.414425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 801/1617] 10.222.0.49 () {54 vars in 962 bytes} [Tue Aug 2 10:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:06.419445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26db97-19db-42c0-b556-ddae36927edf None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.535976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:06.548100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 817/1618] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:06.555765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be086b96-ec78-4748-a255-bcacd5df65ca None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.556327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26db97-19db-42c0-b556-ddae36927edf None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.562791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.564174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.564660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.565094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.656648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.658134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.676389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 802/1619] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:06.678041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.688305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be086b96-ec78-4748-a255-bcacd5df65ca None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.691588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601d255-59d1-465d-af61-596ad885c9ed service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.695743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb47019-23fc-4d85-8925-11c73e1834ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.696393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47019-23fc-4d85-8925-11c73e1834ef service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.696685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47019-23fc-4d85-8925-11c73e1834ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.696967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47019-23fc-4d85-8925-11c73e1834ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.699482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8583c-1ee0-49e9-86fa-3b1e66dc3bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.700510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8583c-1ee0-49e9-86fa-3b1e66dc3bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.701076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8583c-1ee0-49e9-86fa-3b1e66dc3bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.701889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8583c-1ee0-49e9-86fa-3b1e66dc3bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.702836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8583c-1ee0-49e9-86fa-3b1e66dc3bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.706259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8583c-1ee0-49e9-86fa-3b1e66dc3bf2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.714283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb47019-23fc-4d85-8925-11c73e1834ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.716281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb47019-23fc-4d85-8925-11c73e1834ef service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.732069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 818/1620] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:06.733808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d12afd-1194-4a08-a0eb-32fd277aad08 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.734154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d12afd-1194-4a08-a0eb-32fd277aad08 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.734440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d12afd-1194-4a08-a0eb-32fd277aad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.772238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-55d12afd-1194-4a08-a0eb-32fd277aad08 None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:06.773924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-55d12afd-1194-4a08-a0eb-32fd277aad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:06.791940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 803/1621] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:06.793588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567ad2c-9723-4fbf-a0d2-4c875c3e10ea None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.794994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567ad2c-9723-4fbf-a0d2-4c875c3e10ea None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.796646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567ad2c-9723-4fbf-a0d2-4c875c3e10ea None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.805206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 804/1622] 10.222.0.49 () {54 vars in 912 bytes} [Tue Aug 2 10:21:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:06.807460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567ad2c-9723-4fbf-a0d2-4c875c3e10ea None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.833420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567ad2c-9723-4fbf-a0d2-4c875c3e10ea None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.841289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.842285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.842661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.843018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.843990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.847154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.895158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 819/1623] 10.222.0.49 () {58 vars in 1059 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:06.899233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d12afd-1194-4a08-a0eb-32fd277aad08 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.902660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:06.909623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d12afd-1194-4a08-a0eb-32fd277aad08 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.916025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31a030-140e-4a9b-957e-35b634bf3e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:06.916656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a030-140e-4a9b-957e-35b634bf3e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.916938 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a030-140e-4a9b-957e-35b634bf3e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.917213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31a030-140e-4a9b-957e-35b634bf3e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.917917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31a030-140e-4a9b-957e-35b634bf3e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:06.919672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31a030-140e-4a9b-957e-35b634bf3e22 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:06.935865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 820/1624] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:06.935865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77802d-bf23-4166-baa8-38ffc044a62b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:06.935865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77802d-bf23-4166-baa8-38ffc044a62b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:06.935865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77802d-bf23-4166-baa8-38ffc044a62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:06.956952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1d77802d-bf23-4166-baa8-38ffc044a62b None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:06.958295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1d77802d-bf23-4166-baa8-38ffc044a62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:06.970757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 821/1625] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:06.971866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77802d-bf23-4166-baa8-38ffc044a62b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.983911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77802d-bf23-4166-baa8-38ffc044a62b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:06.993450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 805/1626] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:06 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:06.994697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:06.999610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.000340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.000620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.001100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.091807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be919cb4-0684-4738-91d4-9a176bf832d9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:07.097458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.098378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.098949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.099457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] PATH_INFO: `/v3/projects/0e50fa0ee0fd4d8baa9813c113bd43dc/users/f9ee227fa81d4a03b98b609f738c5c47/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.103085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.105142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.123603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 822/1627] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:07.124756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:07.130324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e50fa0ee0fd4d8baa9813c113bd43dc, user_id=f9ee227fa81d4a03b98b609f738c5c47, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.133654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.215521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39f131-81a3-4753-864c-5ec42d880a7a service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:07.239888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 806/1628] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:06 2022] PUT /identity/v3/projects/0e50fa0ee0fd4d8baa9813c113bd43dc/users/f9ee227fa81d4a03b98b609f738c5c47/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:07.239888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:07.260655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:07.336229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.337190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.337382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.337728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.364463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:07.429141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d25fe-ba60-41a5-8437-cecfce77c926 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:07.437840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0a98b-50e4-4645-a0c8-659ca45dec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.438732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0a98b-50e4-4645-a0c8-659ca45dec26 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.439233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0a98b-50e4-4645-a0c8-659ca45dec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.439745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0a98b-50e4-4645-a0c8-659ca45dec26 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.441419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0a98b-50e4-4645-a0c8-659ca45dec26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.444177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0a98b-50e4-4645-a0c8-659ca45dec26 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.481236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.482683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.498422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 823/1629] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:07.499314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:07.510668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440de730-b2e1-4ee6-b203-60b48a20c84e service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:07.517549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8c78-5238-4daa-b1a1-050f77e2c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.518298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8c78-5238-4daa-b1a1-050f77e2c28b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.518691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8c78-5238-4daa-b1a1-050f77e2c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.519142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8c78-5238-4daa-b1a1-050f77e2c28b admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.529879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8c78-5238-4daa-b1a1-050f77e2c28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.529879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8c78-5238-4daa-b1a1-050f77e2c28b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.538094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 824/1630] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:07.540017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41220d5e-b0a2-4433-973b-4743ea5313c6 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.540492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41220d5e-b0a2-4433-973b-4743ea5313c6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.540857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41220d5e-b0a2-4433-973b-4743ea5313c6 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.545482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 825/1631] 10.222.0.49 () {54 vars in 912 bytes} [Tue Aug 2 10:21:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:07.546521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41220d5e-b0a2-4433-973b-4743ea5313c6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:07.559963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41220d5e-b0a2-4433-973b-4743ea5313c6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:07.564434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07e1e8-ef04-4764-af57-7ac79a9699bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.569803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07e1e8-ef04-4764-af57-7ac79a9699bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.569803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07e1e8-ef04-4764-af57-7ac79a9699bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.569803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07e1e8-ef04-4764-af57-7ac79a9699bc admin admin] PATH_INFO: `/v3/projects/0e50fa0ee0fd4d8baa9813c113bd43dc/users/f9ee227fa81d4a03b98b609f738c5c47/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.569803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07e1e8-ef04-4764-af57-7ac79a9699bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e50fa0ee0fd4d8baa9813c113bd43dc, user_id=f9ee227fa81d4a03b98b609f738c5c47, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.570836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07e1e8-ef04-4764-af57-7ac79a9699bc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.584059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 826/1632] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:07 2022] PUT /identity/v3/projects/0e50fa0ee0fd4d8baa9813c113bd43dc/users/f9ee227fa81d4a03b98b609f738c5c47/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:07.585326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4c27-da4a-41be-8367-92e226ac7680 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.585671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4c27-da4a-41be-8367-92e226ac7680 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.585954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4c27-da4a-41be-8367-92e226ac7680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.600580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 807/1633] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:07 2022] POST /identity/v3/users => generated 297 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:07.601672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25483732-6f93-441b-9d43-3c5b1975dded None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.601994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25483732-6f93-441b-9d43-3c5b1975dded None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.602248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25483732-6f93-441b-9d43-3c5b1975dded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.616375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b0af4c27-da4a-41be-8367-92e226ac7680 None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:07.617685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b0af4c27-da4a-41be-8367-92e226ac7680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:07.651755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-25483732-6f93-441b-9d43-3c5b1975dded None None] MFA Rules not processed for user `f9ee227fa81d4a03b98b609f738c5c47`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:07.651996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-25483732-6f93-441b-9d43-3c5b1975dded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:07.760411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 827/1634] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:21:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:07.761050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af4c27-da4a-41be-8367-92e226ac7680 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:07.810392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 808/1635] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:07.829566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25483732-6f93-441b-9d43-3c5b1975dded None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:07.857630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:07.934841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25483732-6f93-441b-9d43-3c5b1975dded None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:07.942128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.942935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.943330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.943670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.945492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af4c27-da4a-41be-8367-92e226ac7680 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:07.955816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.957508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:07.957738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.958766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:07.959754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:07.960276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:07.961166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:07.964604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:07.973762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 809/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:08.104921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 828/1637] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:08.107763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:08.107763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5300e96-b797-4e98-8a08-c0be2a64de1e service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:08.107763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:08.107763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:08.108233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:08.108233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] PATH_INFO: `/v3/projects/58a0e80a077f448596ec7b486a5a6937/users/aacea64ed80a4ae28eb29aaf85ac08e9/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:08.108233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a0e80a077f448596ec7b486a5a6937, user_id=aacea64ed80a4ae28eb29aaf85ac08e9, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:08.108233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:08.188071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 810/1638] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:08 2022] PUT /identity/v3/projects/58a0e80a077f448596ec7b486a5a6937/users/aacea64ed80a4ae28eb29aaf85ac08e9/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:08.192224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:08.206902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734146c-2f60-4d9e-bd7c-de8746c6173a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:08.294745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f1f3-d71c-4d84-8fdb-fbbfae7016f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:08.295617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f1f3-d71c-4d84-8fdb-fbbfae7016f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:08.315451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f1f3-d71c-4d84-8fdb-fbbfae7016f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:08.315451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f1f3-d71c-4d84-8fdb-fbbfae7016f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:08.315451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f1f3-d71c-4d84-8fdb-fbbfae7016f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:08.315451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f1f3-d71c-4d84-8fdb-fbbfae7016f9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:08.315451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 829/1639] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:08.318818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:08.329819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f681a3d-1ddd-401f-886d-5f6f27513a3c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:08.335011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:08.336245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:08.336536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:08.336818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] PATH_INFO: `/v3/projects/58a0e80a077f448596ec7b486a5a6937/users/aacea64ed80a4ae28eb29aaf85ac08e9/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:08.338838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a0e80a077f448596ec7b486a5a6937, user_id=aacea64ed80a4ae28eb29aaf85ac08e9, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:08.341676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:08.353616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 811/1640] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:08 2022] PUT /identity/v3/projects/58a0e80a077f448596ec7b486a5a6937/users/aacea64ed80a4ae28eb29aaf85ac08e9/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:08.365416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054452e-7c19-4e5c-8d29-127f1abe6279 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:08.365676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054452e-7c19-4e5c-8d29-127f1abe6279 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:08.365955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054452e-7c19-4e5c-8d29-127f1abe6279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:08.402597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5054452e-7c19-4e5c-8d29-127f1abe6279 None None] MFA Rules not processed for user `aacea64ed80a4ae28eb29aaf85ac08e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:08.406097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5054452e-7c19-4e5c-8d29-127f1abe6279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:08.515354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 830/1641] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:08.567859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:08.590671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:08.652119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152033d-747c-4bbf-8c70-15436da4ac2a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:08.672017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:08.672809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:08.673102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:08.673383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:09.132572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:09.132572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:09.132572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 812/1642] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:09.135947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054452e-7c19-4e5c-8d29-127f1abe6279 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:09.135947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054452e-7c19-4e5c-8d29-127f1abe6279 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:09.148417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:09.150769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:09.150769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:09.150769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:09.173988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:09.176156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:09.199487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 831/1643] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:21:09.608148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:09.619645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c33db-eb69-4568-ab2f-c622220e65c2 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:09.625911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27805e4f-a075-4670-822f-961b662c4539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:09.626613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27805e4f-a075-4670-822f-961b662c4539 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:09.626903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27805e4f-a075-4670-822f-961b662c4539 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:09.627714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27805e4f-a075-4670-822f-961b662c4539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:09.724801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27805e4f-a075-4670-822f-961b662c4539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:09.728853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27805e4f-a075-4670-822f-961b662c4539 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:09.743080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 813/1644] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:10.382460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:10.392957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdb6ca-7e70-4af4-be00-ce57813f65e1 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:10.400332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:10.401415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:10.401934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:10.402638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:10.444141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:10.477225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:10.478734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:10.518112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 832/1645] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:15.369466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53878133-0f7d-4a1e-8618-7739efba0770 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:15.374049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53878133-0f7d-4a1e-8618-7739efba0770 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:15.374049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53878133-0f7d-4a1e-8618-7739efba0770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:15.439626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-53878133-0f7d-4a1e-8618-7739efba0770 None None] MFA Rules not processed for user `d2ee2b6dd6f647179d2af999b3e62788`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:15.442110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-53878133-0f7d-4a1e-8618-7739efba0770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:15.537018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:15.565099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 814/1646] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:15.579640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:15.672518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c3d8c-e394-4cea-b684-edcebe95c008 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:15.680647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476545c0-a4e3-4bfa-9bf8-488badf8527b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:15.681650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476545c0-a4e3-4bfa-9bf8-488badf8527b service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:15.682132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476545c0-a4e3-4bfa-9bf8-488badf8527b service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:15.682614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476545c0-a4e3-4bfa-9bf8-488badf8527b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:15.698402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476545c0-a4e3-4bfa-9bf8-488badf8527b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:15.702526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476545c0-a4e3-4bfa-9bf8-488badf8527b service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:15.717178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 833/1647] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:17.907791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53878133-0f7d-4a1e-8618-7739efba0770 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:18.053382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53878133-0f7d-4a1e-8618-7739efba0770 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:18.071931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:18.072617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:18.072903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:18.073348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] PATH_INFO: `/v3/users/998e04c59ad64d3e98517ee84c4268a1` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:18.074731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=998e04c59ad64d3e98517ee84c4268a1)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:18.076874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:18.430767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998e04c59ad64d3e98517ee84c4268a1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:18.436937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998e04c59ad64d3e98517ee84c4268a1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:18.462674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998e04c59ad64d3e98517ee84c4268a1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:18.548287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998e04c59ad64d3e98517ee84c4268a1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:18.570261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998e04c59ad64d3e98517ee84c4268a1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:18.588592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 815/1648] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:17 2022] DELETE /identity/v3/users/998e04c59ad64d3e98517ee84c4268a1 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:18.593051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14e525-f966-4de8-aa8e-23677aa0771c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:18.593649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14e525-f966-4de8-aa8e-23677aa0771c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:18.593968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14e525-f966-4de8-aa8e-23677aa0771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:18.622214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2e14e525-f966-4de8-aa8e-23677aa0771c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:18.624014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e14e525-f966-4de8-aa8e-23677aa0771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:18.807147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 834/1649] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:18.856774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:18.858730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:18.958518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:18.958953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:18.959503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:19.057666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] MFA Rules not processed for user `306ea8d439d94da18d43d91ce9838e19`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:19.061535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:19.105592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6c4dc-03a6-441f-92d3-b011f5833007 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:19.127168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0521380-6adc-4a63-b52b-1e3cd69cb1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:19.129189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521380-6adc-4a63-b52b-1e3cd69cb1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:19.131315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521380-6adc-4a63-b52b-1e3cd69cb1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:19.131754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521380-6adc-4a63-b52b-1e3cd69cb1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:19.156526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0521380-6adc-4a63-b52b-1e3cd69cb1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:19.161542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0521380-6adc-4a63-b52b-1e3cd69cb1f0 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:19.193163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 816/1650] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:19.213992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:19.272734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 835/1651] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:19.283857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd1593-e782-4f21-98a1-522fb72b1fcf None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:19.284702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd1593-e782-4f21-98a1-522fb72b1fcf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:19.285017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd1593-e782-4f21-98a1-522fb72b1fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:19.364473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5fdd1593-e782-4f21-98a1-522fb72b1fcf None None] MFA Rules not processed for user `306ea8d439d94da18d43d91ce9838e19`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:19.366289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5fdd1593-e782-4f21-98a1-522fb72b1fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:19.389240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 817/1652] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:19.401468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:19.402970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:19.507975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637d5f5-67ce-4e7c-be50-3a0435b72522 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:19.514117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637d5f5-67ce-4e7c-be50-3a0435b72522 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:19.514117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637d5f5-67ce-4e7c-be50-3a0435b72522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:19.522043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:19.609382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6637d5f5-67ce-4e7c-be50-3a0435b72522 None None] MFA Rules not processed for user `f6880f2f7ad749beb68cd6445f3cc6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:19.633145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6637d5f5-67ce-4e7c-be50-3a0435b72522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:19.679155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f3ee5-104c-4904-8f34-81ff88d7f6dc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:19.703945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:19.704623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:19.704913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:19.705202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:19.728328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:19.732407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:19.758882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 836/1653] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:19.762301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 818/1654] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:19.894154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:19.898781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:19.944958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7529463-f837-4666-97b2-6277c49aebb0 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:19.973750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d823b-b632-4906-9e7c-92407568dcb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:19.974400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d823b-b632-4906-9e7c-92407568dcb7 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:19.974691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d823b-b632-4906-9e7c-92407568dcb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:19.975071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d823b-b632-4906-9e7c-92407568dcb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:19.998238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d823b-b632-4906-9e7c-92407568dcb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:20.000240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d823b-b632-4906-9e7c-92407568dcb7 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:20.026340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 837/1655] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:21:20.574919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dee9-b15b-4989-807b-27e1a8c6c32b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:20.577553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dee9-b15b-4989-807b-27e1a8c6c32b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:20.577553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dee9-b15b-4989-807b-27e1a8c6c32b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:20.577553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 819/1656] 10.222.0.49 () {54 vars in 912 bytes} [Tue Aug 2 10:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:20.590458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1ec3-289b-4be2-a123-5560141de6f9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:20.590458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1ec3-289b-4be2-a123-5560141de6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:20.590458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1ec3-289b-4be2-a123-5560141de6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:20.680029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8dee9-b15b-4989-807b-27e1a8c6c32b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:20.680580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-89f8dee9-b15b-4989-807b-27e1a8c6c32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:20.681886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-599f1ec3-289b-4be2-a123-5560141de6f9 None None] MFA Rules not processed for user `09cabb184e704aecbb0a3bfb1cd0a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:20.683780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-599f1ec3-289b-4be2-a123-5560141de6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:20.720584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8dee9-b15b-4989-807b-27e1a8c6c32b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:20.728153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 838/1657] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:20.728444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:20.729462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:20.730858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:20.731441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:20.732399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] PATH_INFO: `/v3/projects/8da396d25bf44b8d89ab2f28f0953db4` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:20.732866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:20.733975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:20.735763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da396d25bf44b8d89ab2f28f0953db4)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:20.741257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:21:20.741257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: warnings.warn(msg) Aug 02 10:21:20.741257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: Aug 02 10:21:20.744366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:20.793984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] MFA Rules not processed for user `d2ee2b6dd6f647179d2af999b3e62788`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:20.796733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:20.950907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da396d25bf44b8d89ab2f28f0953db4', 'request_id': 'req-643eacd4-222d-4f7f-a9f4-7cff5b59c269'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:20.965672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 839/1658] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:20.968254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:20.977227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:20.979139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da396d25bf44b8d89ab2f28f0953db4', 'request_id': 'req-643eacd4-222d-4f7f-a9f4-7cff5b59c269'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:20.999906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec590d6-04b1-407a-9ba0-a74eb3661ee5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:21.068912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da396d25bf44b8d89ab2f28f0953db4', 'request_id': 'req-643eacd4-222d-4f7f-a9f4-7cff5b59c269'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:21.089385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] The token cache is being invalidate because project 8da396d25bf44b8d89ab2f28f0953db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:21.089688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:21.094571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-643eacd4-222d-4f7f-a9f4-7cff5b59c269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:21.111300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 820/1659] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:20 2022] DELETE /identity/v3/projects/8da396d25bf44b8d89ab2f28f0953db4 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:21.114417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52de51-5f73-4d3e-b090-108ab0795c99 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.120513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52de51-5f73-4d3e-b090-108ab0795c99 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.120513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52de51-5f73-4d3e-b090-108ab0795c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.138705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:21.155068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6c52de51-5f73-4d3e-b090-108ab0795c99 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:21.156769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6c52de51-5f73-4d3e-b090-108ab0795c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:21.166391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d449-9081-42d7-8d7b-e310f1f3a8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:21.167378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d449-9081-42d7-8d7b-e310f1f3a8ba service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.173021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d449-9081-42d7-8d7b-e310f1f3a8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.173383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d449-9081-42d7-8d7b-e310f1f3a8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.185777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1957d449-9081-42d7-8d7b-e310f1f3a8ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:21.239708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957d449-9081-42d7-8d7b-e310f1f3a8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:21.241203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957d449-9081-42d7-8d7b-e310f1f3a8ba service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:21.265331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 840/1660] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:21.321195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 821/1661] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:21.328919 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9efcb-af49-455f-b266-b28337138ac9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.330278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9efcb-af49-455f-b266-b28337138ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.330278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9efcb-af49-455f-b266-b28337138ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.355876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-56f9efcb-af49-455f-b266-b28337138ac9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:21.357404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-56f9efcb-af49-455f-b266-b28337138ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:21.371681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52de51-5f73-4d3e-b090-108ab0795c99 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:21.372547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 841/1662] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:21.386429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52de51-5f73-4d3e-b090-108ab0795c99 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:21.395301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:21.395977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.397661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.399205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.401571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:21.402078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:21.405683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:21.420958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 822/1663] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:21.425245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9efcb-af49-455f-b266-b28337138ac9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:21.435573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9efcb-af49-455f-b266-b28337138ac9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:21.457788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c9071-be9a-4e02-97e0-2359e7c83b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:21.458736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c9071-be9a-4e02-97e0-2359e7c83b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.459132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c9071-be9a-4e02-97e0-2359e7c83b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.460791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c9071-be9a-4e02-97e0-2359e7c83b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.462129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c9071-be9a-4e02-97e0-2359e7c83b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:21.464446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c9071-be9a-4e02-97e0-2359e7c83b81 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:21.625593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 842/1664] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:21 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:21.630041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:21.638043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:21.643138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bab14-625f-41b4-9d09-a560ce922f5f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:21.765390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77368751-521c-4f19-b704-9ca99b9c8c9d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.765930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77368751-521c-4f19-b704-9ca99b9c8c9d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.766155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77368751-521c-4f19-b704-9ca99b9c8c9d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.766950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:21.767798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.768042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 843/1665] 10.222.0.49 () {54 vars in 912 bytes} [Tue Aug 2 10:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:21:21.769136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.769429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.771112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:21.773745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:21.774651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baee1a-9e2e-4d55-8297-46098823dc3d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.775129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baee1a-9e2e-4d55-8297-46098823dc3d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.775651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baee1a-9e2e-4d55-8297-46098823dc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.802577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-74baee1a-9e2e-4d55-8297-46098823dc3d None None] MFA Rules not processed for user `09cabb184e704aecbb0a3bfb1cd0a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:21.804034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-74baee1a-9e2e-4d55-8297-46098823dc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:21.884278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 844/1666] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:21:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:21.894894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 823/1667] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:21 2022] POST /identity/v3/users => generated 287 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:21.900305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baee1a-9e2e-4d55-8297-46098823dc3d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:21.901894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:21.903824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:21.922694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baee1a-9e2e-4d55-8297-46098823dc3d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:21.930076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77141a-97fd-404d-91a7-bbcb18098a37 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:21.930498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:21.931156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.931481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.931763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.932494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:21.934245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:21.937442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:21.938280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.938693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.939269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:21.953337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 845/1668] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:21.965816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:21.971275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:21.979958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:21.994960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d16bf6-2c29-4ead-9d5c-077d64ea3920 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:21.998475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e08c2-e609-4189-8241-11d84e30fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:21.999287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e08c2-e609-4189-8241-11d84e30fd2c admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:21.999495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e08c2-e609-4189-8241-11d84e30fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:21.999775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e08c2-e609-4189-8241-11d84e30fd2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.000488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e08c2-e609-4189-8241-11d84e30fd2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:22.002281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e08c2-e609-4189-8241-11d84e30fd2c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:22.013261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 824/1669] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:22.016099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 846/1670] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:22.021080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:22.036257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86168b-a149-4d3c-8a7c-c43f54205bde service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:22.041961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc3a9-4c56-4b68-8289-c7adebecfdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:22.042797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc3a9-4c56-4b68-8289-c7adebecfdec admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.043015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc3a9-4c56-4b68-8289-c7adebecfdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.043371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc3a9-4c56-4b68-8289-c7adebecfdec admin admin] PATH_INFO: `/v3/projects/e40b9752eb744ef5a52dbbca8c2dca75/users/1615461b95234f4a8a67439b06cd5e20/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.075972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dc3a9-4c56-4b68-8289-c7adebecfdec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40b9752eb744ef5a52dbbca8c2dca75, user_id=1615461b95234f4a8a67439b06cd5e20, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:22.079027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dc3a9-4c56-4b68-8289-c7adebecfdec admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:22.157304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 825/1671] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:22 2022] PUT /identity/v3/projects/e40b9752eb744ef5a52dbbca8c2dca75/users/1615461b95234f4a8a67439b06cd5e20/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:21:22.169022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd86251-dc38-478b-8d60-ce9f2c600842 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.169500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd86251-dc38-478b-8d60-ce9f2c600842 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.169975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd86251-dc38-478b-8d60-ce9f2c600842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.256004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8cd86251-dc38-478b-8d60-ce9f2c600842 None None] MFA Rules not processed for user `1615461b95234f4a8a67439b06cd5e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:22.258112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8cd86251-dc38-478b-8d60-ce9f2c600842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:22.275375 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb564-df12-4cb9-bb2f-1fe46bfae4f8 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.275674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb564-df12-4cb9-bb2f-1fe46bfae4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.275958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb564-df12-4cb9-bb2f-1fe46bfae4f8 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.277649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 826/1672] 10.222.0.49 () {54 vars in 961 bytes} [Tue Aug 2 10:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:22.290235 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79100762-a4a5-4288-ba27-7d9038b294fa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.290483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79100762-a4a5-4288-ba27-7d9038b294fa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.290774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79100762-a4a5-4288-ba27-7d9038b294fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.344424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-79100762-a4a5-4288-ba27-7d9038b294fa None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:22.346118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-79100762-a4a5-4288-ba27-7d9038b294fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:22.394529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 847/1673] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:22.395697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd86251-dc38-478b-8d60-ce9f2c600842 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:22.445271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 827/1674] 10.222.0.49 () {58 vars in 1058 bytes} [Tue Aug 2 10:21:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:22.453617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:22.454885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79100762-a4a5-4288-ba27-7d9038b294fa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:22.473817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79100762-a4a5-4288-ba27-7d9038b294fa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:22.486841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:22.488384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.488816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.489931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.491951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:22.519914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd86251-dc38-478b-8d60-ce9f2c600842 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:22.523446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:22.525319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:22.539641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:22.540298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.540584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.540861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.541534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 828/1675] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:22.541651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:22.543914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:22.693075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 848/1676] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:22 2022] POST /identity/v3/projects => generated 373 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:22.694372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:22.705698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5f0d7-fee7-4fdf-8258-053115cb9858 service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:22.749051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:22.767999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:22.769170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.769514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.769762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.770429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:22.772456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:22.911067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 829/1677] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:22 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:22.924696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:22.943001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac73dd-30e7-4a63-8458-98b5ef60cd12 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:22.949507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:22.950162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:22.950720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:22.951036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:22.951917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:22.954063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:22.971120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 849/1678] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:22.975014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:22.986087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958ee8c-54c6-4317-b16d-cbf0d39558d7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:23.002070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:23.003305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:23.003595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:23.006122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] PATH_INFO: `/v3/projects/e04258320191491a9454d8e2f08bce2a/users/b6d0f15e03764db897d46f4d3071dadc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:23.042592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04258320191491a9454d8e2f08bce2a, user_id=b6d0f15e03764db897d46f4d3071dadc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:23.051112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:23.150927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 830/1679] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:22 2022] PUT /identity/v3/projects/e04258320191491a9454d8e2f08bce2a/users/b6d0f15e03764db897d46f4d3071dadc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:23.154851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:23.169249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519e7fa-c0fd-405d-873b-1b638e097823 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:23.312691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b23f4e-0e9d-4371-89a4-a651e9341215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:23.315983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23f4e-0e9d-4371-89a4-a651e9341215 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:23.316552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23f4e-0e9d-4371-89a4-a651e9341215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:23.317258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23f4e-0e9d-4371-89a4-a651e9341215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:23.318276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b23f4e-0e9d-4371-89a4-a651e9341215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:23.322370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b23f4e-0e9d-4371-89a4-a651e9341215 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:23.369112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:23.369863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 850/1680] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:23.394483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412c5a9-e65e-4147-8778-66900bf3a190 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:23.421026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:23.422411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:23.422928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:23.423510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] PATH_INFO: `/v3/projects/e04258320191491a9454d8e2f08bce2a/users/b6d0f15e03764db897d46f4d3071dadc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:23.427983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04258320191491a9454d8e2f08bce2a, user_id=b6d0f15e03764db897d46f4d3071dadc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:23.430793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:23.447818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 831/1681] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:23 2022] PUT /identity/v3/projects/e04258320191491a9454d8e2f08bce2a/users/b6d0f15e03764db897d46f4d3071dadc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:23.460203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7280d50-e168-4e84-a82a-1d4eb4a47bac None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:23.461368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7280d50-e168-4e84-a82a-1d4eb4a47bac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:23.461871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7280d50-e168-4e84-a82a-1d4eb4a47bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:23.555254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a7280d50-e168-4e84-a82a-1d4eb4a47bac None None] MFA Rules not processed for user `b6d0f15e03764db897d46f4d3071dadc`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:23.555254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a7280d50-e168-4e84-a82a-1d4eb4a47bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:23.703328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 851/1682] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:23.716696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:23.719134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:23.814824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443442d-9840-4046-b5c8-f8c304bd6b15 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:23.818581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:23.819268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:23.819714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:23.819992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:23.839496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:23.840975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:23.852334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 832/1683] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:24.457948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:24.458716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:24.458716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:24.482591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] MFA Rules not processed for user `f6880f2f7ad749beb68cd6445f3cc6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:24.488299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:24.531576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:24.621799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 852/1684] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:25.885008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:25.945209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:25.945209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:26.032840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6d67-7275-42fd-b5db-16e584e837ec service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:26.043648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:26.044319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:26.044633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:26.044910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] PATH_INFO: `/v3/users/1615461b95234f4a8a67439b06cd5e20` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:26.048724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1615461b95234f4a8a67439b06cd5e20)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:26.051286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:26.178407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09246e14-20ee-4e7d-b6ff-e5f92e839cd9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:26.201727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:26.202644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:26.203048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:26.203590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] PATH_INFO: `/v3/users/04456e2f28b14ff28857b7d22b4a468e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:26.205267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04456e2f28b14ff28857b7d22b4a468e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:26.208618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:21:26.208618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: warnings.warn(msg) Aug 02 10:21:26.208618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: Aug 02 10:21:26.210881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:26.397646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:26.415703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1615461b95234f4a8a67439b06cd5e20'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.423694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1615461b95234f4a8a67439b06cd5e20'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.438187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1615461b95234f4a8a67439b06cd5e20'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.520203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1615461b95234f4a8a67439b06cd5e20'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.528222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04456e2f28b14ff28857b7d22b4a468e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.548788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04456e2f28b14ff28857b7d22b4a468e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.550082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04456e2f28b14ff28857b7d22b4a468e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.552493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04456e2f28b14ff28857b7d22b4a468e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.557791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-32da4db5-4bb8-4377-93ce-3fd155222dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1615461b95234f4a8a67439b06cd5e20'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.569236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 833/1685] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:25 2022] DELETE /identity/v3/users/1615461b95234f4a8a67439b06cd5e20 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:26.573656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:26.574004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:26.574290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:26.591894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-47e9d5e4-412e-4388-85ae-aad22724c9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04456e2f28b14ff28857b7d22b4a468e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:26.624476 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:26.625726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:26.723872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 853/1686] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:25 2022] DELETE /identity/v3/users/04456e2f28b14ff28857b7d22b4a468e => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:26.729640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b21e8-0273-47e0-8c63-a59d0831c953 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:26.730090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b21e8-0273-47e0-8c63-a59d0831c953 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:26.730469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b21e8-0273-47e0-8c63-a59d0831c953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:26.803398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-eb1b21e8-0273-47e0-8c63-a59d0831c953 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:26.810924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 834/1687] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:26.830459 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:26.832832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:26.838787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eb1b21e8-0273-47e0-8c63-a59d0831c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:26.999097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ee012-bdec-467d-b0dd-6104c4a406dd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:27.001623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:27.029073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 854/1688] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:27.040308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b21e8-0273-47e0-8c63-a59d0831c953 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:27.103569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75d7af-328f-4ce2-8bd4-5a76053a17bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:27.104859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75d7af-328f-4ce2-8bd4-5a76053a17bd service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:27.105347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75d7af-328f-4ce2-8bd4-5a76053a17bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:27.105843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75d7af-328f-4ce2-8bd4-5a76053a17bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:27.122356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75d7af-328f-4ce2-8bd4-5a76053a17bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:27.124523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75d7af-328f-4ce2-8bd4-5a76053a17bd service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:27.148240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 835/1689] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:27.161634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b21e8-0273-47e0-8c63-a59d0831c953 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:27.181575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:27.185520 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:27.185808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:27.190568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:27.213460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:27.215720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:27.232680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 855/1690] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:21:27.810377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:27.810886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:27.811402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:27.867592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:27.903498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:27.905112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:27.956991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b7d5d-0217-445e-bada-9f1e4bf3c4c9 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:27.981616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:27.993973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:27.994799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:27.995097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:27.995397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] PATH_INFO: `/v3/users/196de827124f47c887637b59e7e346ea` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:27.996291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:28.005602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] RBAC: Authorizing `identity:delete_user(user_id=196de827124f47c887637b59e7e346ea)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:28.008178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:28.237705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 836/1691] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:28.410518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196de827124f47c887637b59e7e346ea'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:28.424345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196de827124f47c887637b59e7e346ea'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:28.425915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196de827124f47c887637b59e7e346ea'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:28.427048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196de827124f47c887637b59e7e346ea'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:28.456336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8f0670c1-ab4f-42bc-93ec-69e63e0e16de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196de827124f47c887637b59e7e346ea'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:28.626206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:28.628880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:28.641844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 856/1692] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:27 2022] DELETE /identity/v3/users/196de827124f47c887637b59e7e346ea => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:28.649023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:28.649365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:28.649710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:28.682575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:28.684021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:28.689441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420c68c-3e59-4ac3-8fcb-8fd042079646 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:28.770467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:28.771437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:28.771871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:28.771988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] PATH_INFO: `/v3/projects/5db81d6f81114ce6a527b6e718453fc5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:28.773297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db81d6f81114ce6a527b6e718453fc5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:28.776454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 857/1693] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:28.778057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:28.780436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:28.786226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:28.917578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:28.963664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aef2ad-fd06-4214-9543-261659ea8e47 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:28.992184 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:28.992874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:28.993164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:28.993437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:29.005714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:29.037015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db81d6f81114ce6a527b6e718453fc5', 'request_id': 'req-9c7723ac-96f4-4673-80de-b98c77a13e6c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.061214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db81d6f81114ce6a527b6e718453fc5', 'request_id': 'req-9c7723ac-96f4-4673-80de-b98c77a13e6c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.146000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:29.154472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db81d6f81114ce6a527b6e718453fc5', 'request_id': 'req-9c7723ac-96f4-4673-80de-b98c77a13e6c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.162363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:29.183944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 858/1694] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:21:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:21:29.185489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:29.185860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] The token cache is being invalidate because project 5db81d6f81114ce6a527b6e718453fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:29.186479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.187173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:29.188826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.195564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 837/1695] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:28 2022] DELETE /identity/v3/projects/5db81d6f81114ce6a527b6e718453fc5 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:29.195564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:29.197993 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:29.307969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb394e-22fa-44bc-aaf5-7f60419f77d2 service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:29.367717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7723ac-96f4-4673-80de-b98c77a13e6c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:29.385026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:29.386096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:29.386463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:29.386849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] PATH_INFO: `/v3/projects/e40b9752eb744ef5a52dbbca8c2dca75` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:29.388199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40b9752eb744ef5a52dbbca8c2dca75)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:29.390777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:29.452898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9646d87-8868-4ff5-9f5d-0a289cd12381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:29.453395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9646d87-8868-4ff5-9f5d-0a289cd12381 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:29.453684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9646d87-8868-4ff5-9f5d-0a289cd12381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:29.457954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9646d87-8868-4ff5-9f5d-0a289cd12381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:29.465198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9646d87-8868-4ff5-9f5d-0a289cd12381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:29.501719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9646d87-8868-4ff5-9f5d-0a289cd12381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:29.504280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9646d87-8868-4ff5-9f5d-0a289cd12381 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:29.536556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 859/1696] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:29.553431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0791-c4a5-4f16-ba6d-cae1f41e0b4a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:29.554851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0791-c4a5-4f16-ba6d-cae1f41e0b4a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:29.556951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0791-c4a5-4f16-ba6d-cae1f41e0b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:29.631775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-30ea0791-c4a5-4f16-ba6d-cae1f41e0b4a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:29.635750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-30ea0791-c4a5-4f16-ba6d-cae1f41e0b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:29.669442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40b9752eb744ef5a52dbbca8c2dca75', 'request_id': 'req-97e61903-5e7a-4602-8f3e-2971e85ae2fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.702362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40b9752eb744ef5a52dbbca8c2dca75', 'request_id': 'req-97e61903-5e7a-4602-8f3e-2971e85ae2fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.895101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 860/1697] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:29.904551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a326630-86ce-400f-b394-3caa529bb8d7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:29.904878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a326630-86ce-400f-b394-3caa529bb8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:29.905183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a326630-86ce-400f-b394-3caa529bb8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:29.907998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40b9752eb744ef5a52dbbca8c2dca75', 'request_id': 'req-97e61903-5e7a-4602-8f3e-2971e85ae2fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.937723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] The token cache is being invalidate because project e40b9752eb744ef5a52dbbca8c2dca75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:29.938076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.938807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:29.961325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 838/1698] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:29 2022] DELETE /identity/v3/projects/e40b9752eb744ef5a52dbbca8c2dca75 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:29.962632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:29.966446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:29.978431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1a326630-86ce-400f-b394-3caa529bb8d7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:29.981869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a326630-86ce-400f-b394-3caa529bb8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:29.985885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:30.011051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 861/1699] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:30.012802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f81ac7-fb69-425e-975d-3084fef905d7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:30.013216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f81ac7-fb69-425e-975d-3084fef905d7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:30.013607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f81ac7-fb69-425e-975d-3084fef905d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:30.030426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e61903-5e7a-4602-8f3e-2971e85ae2fe admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:30.047521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:30.048224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:30.048517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:30.048804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] PATH_INFO: `/v3/users/b6d0f15e03764db897d46f4d3071dadc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:30.054589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d0f15e03764db897d46f4d3071dadc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:30.056686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:30.060365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f2f81ac7-fb69-425e-975d-3084fef905d7 None None] MFA Rules not processed for user `09cabb184e704aecbb0a3bfb1cd0a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:30.062074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f2f81ac7-fb69-425e-975d-3084fef905d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:30.237965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 862/1700] 10.222.0.89 () {58 vars in 1015 bytes} [Tue Aug 2 10:21:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:30.237965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f81ac7-fb69-425e-975d-3084fef905d7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:30.305528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f81ac7-fb69-425e-975d-3084fef905d7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:30.315744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:30.316613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:30.316972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:30.319248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:30.320416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:30.327095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:30.329220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:30.365085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 863/1701] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:30.371297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:30.409970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7e44a-1af6-4dab-93c1-35bcad1a79e5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:30.451164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:30.454667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:30.459030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:30.461676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:30.463785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d0f15e03764db897d46f4d3071dadc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:30.465469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:30.466024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d0f15e03764db897d46f4d3071dadc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:30.474293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:30.495712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d0f15e03764db897d46f4d3071dadc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:30.658138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d0f15e03764db897d46f4d3071dadc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:30.744011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 864/1702] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:30 2022] POST /identity/v3/projects => generated 363 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:30.744011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:30.744011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:30.744454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d0f15e03764db897d46f4d3071dadc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:30.744454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 839/1703] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:29 2022] DELETE /identity/v3/users/b6d0f15e03764db897d46f4d3071dadc => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:30.744454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:30.750702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:30.821472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fdd2b-d89e-432b-b257-a798b6139507 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:30.974538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:30.974538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:30.974538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:30.974538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:30.976738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:30.976738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:30.976738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:31.005151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 865/1704] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:31.008677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:31.015277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:31.079046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cb893-a67b-4d94-97af-4b74347f4827 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:31.092521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa4c7f-d455-4306-8da3-a5cf56c4b028 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:31.093543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:31.110801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:31.111663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:31.112448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:31.113526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:31.116942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:31.141318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:31.142150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:31.142545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:31.143014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] PATH_INFO: `/v3/users/8cbff7e82e0443148d2574deb75ac384` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:31.147597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbff7e82e0443148d2574deb75ac384)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:31.149852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:31.327166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 840/1705] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:30 2022] POST /identity/v3/users => generated 282 bytes in 583 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:31.328557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:31.330675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:31.346284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59af64-4c50-4376-9cd5-6f62df408fe6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:31.351857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:31.352997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:31.353611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:31.354144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] PATH_INFO: `/v3/projects/855d0942718247168191f1760fa678b2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:31.356642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] RBAC: Authorizing `identity:delete_project(project_id=855d0942718247168191f1760fa678b2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:31.360221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:31.423259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:31.579028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbff7e82e0443148d2574deb75ac384'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.612456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '855d0942718247168191f1760fa678b2', 'request_id': 'req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.623825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbff7e82e0443148d2574deb75ac384'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.625328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbff7e82e0443148d2574deb75ac384'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.633018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbff7e82e0443148d2574deb75ac384'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.639037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '855d0942718247168191f1760fa678b2', 'request_id': 'req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.681316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46bdfbe7-2d37-4074-9986-2e81ea7184d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbff7e82e0443148d2574deb75ac384'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.817928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '855d0942718247168191f1760fa678b2', 'request_id': 'req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.834013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] The token cache is being invalidate because project 855d0942718247168191f1760fa678b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:31.834537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.835467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:31.841569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 841/1706] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:31 2022] DELETE /identity/v3/projects/855d0942718247168191f1760fa678b2 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:31.842940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:31.845191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:31.887711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 866/1707] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:31 2022] DELETE /identity/v3/users/8cbff7e82e0443148d2574deb75ac384 => generated 0 bytes in 882 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:31.893463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ef32-c3da-4d79-8949-65d21990982a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:31.893780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ef32-c3da-4d79-8949-65d21990982a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:31.894028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ef32-c3da-4d79-8949-65d21990982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:31.926873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dc65ef32-c3da-4d79-8949-65d21990982a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:31.928509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dc65ef32-c3da-4d79-8949-65d21990982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:31.977338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daafa3-f477-4732-ab0f-47d5b9d4c22a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:32.031550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 867/1708] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:32.034379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4745395-5432-456f-9641-9ba86b76bfa2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.034908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4745395-5432-456f-9641-9ba86b76bfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.035418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4745395-5432-456f-9641-9ba86b76bfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.083564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d4745395-5432-456f-9641-9ba86b76bfa2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:32.086530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:32.088179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d4745395-5432-456f-9641-9ba86b76bfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:32.107679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 868/1709] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:32.114133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.118005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.118545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.123582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:32.124448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.124937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.125228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.126260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:32.130066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:32.158163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 842/1710] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:32.167343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:32.169377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:32.170798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:32.184126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc7d53-5aea-45a0-969b-5fcb46c574bf admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:32.192187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 869/1711] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:32.193976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:32.194327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:32.196003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.196003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.196003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.197317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:32.205057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:32.211531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:32.243458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 843/1712] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:32.243458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:32.280692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d2405-cafd-4046-8674-e356062972b7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:32.301219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:32.302739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.303748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.305558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.307694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:32.312265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:32.312265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:32.337427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:32.352580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb5375-a90f-4595-8b3f-b0a442fa7ebe None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:32.360167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 844/1713] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:32.361239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:32.365321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:32.369918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.369918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.369918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.374234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ee794-dabd-41fa-a664-af6caa075fa6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:32.382457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:32.386455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.387569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.388313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] PATH_INFO: `/v3/projects/07b7f2d0b8744522b90de1dd4946a977/users/b8f6775d15e9479bb25ea9f83346872d/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.407013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:32.408555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:32.434930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 870/1714] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:32.436829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:32.439611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b7f2d0b8744522b90de1dd4946a977, user_id=b8f6775d15e9479bb25ea9f83346872d, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:32.439737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:32.448147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:32.463913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbeedf-2c43-4fb7-ae38-80fa7d720a53 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:32.476012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:32.476636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.476931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.477277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] PATH_INFO: `/v3/projects/e04258320191491a9454d8e2f08bce2a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.479590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04258320191491a9454d8e2f08bce2a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:32.485605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:32.534186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 845/1715] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:32 2022] PUT /identity/v3/projects/07b7f2d0b8744522b90de1dd4946a977/users/b8f6775d15e9479bb25ea9f83346872d/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:32.537520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:32.753699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69847d7e-bf34-42f4-b97a-4c045585ad0f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:32.774067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce2066-74d5-46bb-be3c-83f4f7b7bdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:32.777628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2066-74d5-46bb-be3c-83f4f7b7bdde admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:32.780373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2066-74d5-46bb-be3c-83f4f7b7bdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:32.780373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2066-74d5-46bb-be3c-83f4f7b7bdde admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:32.780373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce2066-74d5-46bb-be3c-83f4f7b7bdde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:32.785103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce2066-74d5-46bb-be3c-83f4f7b7bdde admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:32.832573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04258320191491a9454d8e2f08bce2a', 'request_id': 'req-aea52acd-a734-4a17-bebe-493995c19c40'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:32.928531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:32.983836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04258320191491a9454d8e2f08bce2a', 'request_id': 'req-aea52acd-a734-4a17-bebe-493995c19c40'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:32.998768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04258320191491a9454d8e2f08bce2a', 'request_id': 'req-aea52acd-a734-4a17-bebe-493995c19c40'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:33.016270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] The token cache is being invalidate because project e04258320191491a9454d8e2f08bce2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:33.017225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:33.017806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 846/1716] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:32 2022] POST /identity/v3/projects => generated 405 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:33.022387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:33.024622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06912b-07e9-4021-82ae-4ac32f70c191 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:33.030214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06912b-07e9-4021-82ae-4ac32f70c191 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:33.030348 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 871/1717] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:32 2022] DELETE /identity/v3/projects/e04258320191491a9454d8e2f08bce2a => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:33.030438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06912b-07e9-4021-82ae-4ac32f70c191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:33.030533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:33.030533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:33.118572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cc06912b-07e9-4021-82ae-4ac32f70c191 None None] MFA Rules not processed for user `b8f6775d15e9479bb25ea9f83346872d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:33.130385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cc06912b-07e9-4021-82ae-4ac32f70c191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:33.306888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea52acd-a734-4a17-bebe-493995c19c40 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:33.320978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 847/1718] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:33.322718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572e14-d54c-4e1b-b835-4e26b88c99ae None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:33.322941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572e14-d54c-4e1b-b835-4e26b88c99ae None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:33.323310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572e14-d54c-4e1b-b835-4e26b88c99ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:33.380200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-49572e14-d54c-4e1b-b835-4e26b88c99ae None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:33.381728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-49572e14-d54c-4e1b-b835-4e26b88c99ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:33.417482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:33.418150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:33.418468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:33.418747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:33.419469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:33.421522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:33.536997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 872/1719] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:33 2022] POST /identity/v3/users => generated 303 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:33.538154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:33.547843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:33.567706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 848/1720] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:33.568824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572e14-d54c-4e1b-b835-4e26b88c99ae None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:33.574616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f8a98-e4b1-4585-980b-9460f5fc2321 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:33.583139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:33.586168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:33.586168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:33.586168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:33.586168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:33.586892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:33.595616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572e14-d54c-4e1b-b835-4e26b88c99ae None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:33.603179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cab9f-de15-4a41-992d-2acafe68b27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:33.604313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cab9f-de15-4a41-992d-2acafe68b27c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:33.606060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cab9f-de15-4a41-992d-2acafe68b27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:33.606060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cab9f-de15-4a41-992d-2acafe68b27c admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:33.632104 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cab9f-de15-4a41-992d-2acafe68b27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:33.633893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cab9f-de15-4a41-992d-2acafe68b27c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:33.658896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 849/1721] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:33.700703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391478ba-aeed-4e9b-9c30-926fcbcb6696 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:33.700703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391478ba-aeed-4e9b-9c30-926fcbcb6696 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:33.700703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391478ba-aeed-4e9b-9c30-926fcbcb6696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:33.721092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-391478ba-aeed-4e9b-9c30-926fcbcb6696 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:33.723559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-391478ba-aeed-4e9b-9c30-926fcbcb6696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:33.746205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 850/1722] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:33.763494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 873/1723] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:33 2022] POST /identity/v3/projects => generated 363 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:33.765289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391478ba-aeed-4e9b-9c30-926fcbcb6696 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:33.766630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:33.802776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391478ba-aeed-4e9b-9c30-926fcbcb6696 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:33.972732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac84b3-fe58-4647-8114-80b248e7b5f4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:33.985757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:33.986463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:33.987371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:33.987697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:33.988414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:33.988726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:33.992870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:34.015389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcf7af-ff68-4508-aa4c-d92d0098cd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:34.016424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcf7af-ff68-4508-aa4c-d92d0098cd26 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.016573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcf7af-ff68-4508-aa4c-d92d0098cd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.017153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcf7af-ff68-4508-aa4c-d92d0098cd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.018016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcf7af-ff68-4508-aa4c-d92d0098cd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:34.019839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcf7af-ff68-4508-aa4c-d92d0098cd26 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:34.036698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 874/1724] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:34.036698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:34.048950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 851/1725] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:34.054362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d470911-c9ef-4b49-b2cf-a14a517e8a27 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.055022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d470911-c9ef-4b49-b2cf-a14a517e8a27 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.055623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d470911-c9ef-4b49-b2cf-a14a517e8a27 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.058102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 852/1726] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:21:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:34.059718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d470911-c9ef-4b49-b2cf-a14a517e8a27 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:34.092949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d470911-c9ef-4b49-b2cf-a14a517e8a27 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:34.108318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f605b-34ba-44d4-ae2b-c7139005e574 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:34.109481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:34.111068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.111555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.113921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.114936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:34.119918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:34.133442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:34.134763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcb179-9b2e-4274-ae08-e5b8eb74b355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:34.136986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb179-9b2e-4274-ae08-e5b8eb74b355 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.137275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb179-9b2e-4274-ae08-e5b8eb74b355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.138169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb179-9b2e-4274-ae08-e5b8eb74b355 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.144630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcb179-9b2e-4274-ae08-e5b8eb74b355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:34.152079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcb179-9b2e-4274-ae08-e5b8eb74b355 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:34.254436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 853/1727] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:34 2022] POST /identity/v3/projects => generated 367 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:34.256151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:34.282118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 875/1728] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:34 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:34.284524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125a6d6-8efd-4939-b96b-94f1f6a4da44 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.285064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125a6d6-8efd-4939-b96b-94f1f6a4da44 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.285897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125a6d6-8efd-4939-b96b-94f1f6a4da44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.335496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c125a6d6-8efd-4939-b96b-94f1f6a4da44 None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:34.339762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c125a6d6-8efd-4939-b96b-94f1f6a4da44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:34.461847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b849de-772e-4e51-b05a-b54a5e95b557 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:34.474720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:34.478343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:34.479426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.486934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.486934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] PATH_INFO: `/v3/projects/b9a08a24e6ae44f998fe04fe5dc5118a/users/14f65eca2b994097aefbfd5c80f4ba52/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.522188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a08a24e6ae44f998fe04fe5dc5118a, user_id=14f65eca2b994097aefbfd5c80f4ba52, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:34.531974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:34.608420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 876/1729] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:34.613584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba3096-541b-48a2-82b7-559c39048004 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.614222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba3096-541b-48a2-82b7-559c39048004 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.614766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba3096-541b-48a2-82b7-559c39048004 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.617420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 877/1730] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:21:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:34.618976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba3096-541b-48a2-82b7-559c39048004 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:34.621460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8cba3096-541b-48a2-82b7-559c39048004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:34.662321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba3096-541b-48a2-82b7-559c39048004 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:34.670021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 854/1731] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:34 2022] PUT /identity/v3/projects/b9a08a24e6ae44f998fe04fe5dc5118a/users/14f65eca2b994097aefbfd5c80f4ba52/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:34.675733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:34.794801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:34.795620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.795915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.796551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] PATH_INFO: `/v3/projects/c217a6065f9e4a77a573a915c372b4ea` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.798023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c217a6065f9e4a77a573a915c372b4ea)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:34.802377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:34.915841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ba405-5445-428f-882f-ccc0de17d0e5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:34.934824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:34.935608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:34.938066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:34.939985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:34.941295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:34.951439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:35.017320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c217a6065f9e4a77a573a915c372b4ea', 'request_id': 'req-2ff87136-f2b9-4738-a90e-05c0f7cbf918'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:35.142776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c217a6065f9e4a77a573a915c372b4ea', 'request_id': 'req-2ff87136-f2b9-4738-a90e-05c0f7cbf918'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:35.149609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c217a6065f9e4a77a573a915c372b4ea', 'request_id': 'req-2ff87136-f2b9-4738-a90e-05c0f7cbf918'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:35.156700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 855/1732] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:34 2022] POST /identity/v3/users => generated 284 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:35.159426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:35.184703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae880b1-11f5-4631-944c-9c910ee17a26 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:35.197376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] The token cache is being invalidate because project c217a6065f9e4a77a573a915c372b4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:35.197677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:35.199975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ff87136-f2b9-4738-a90e-05c0f7cbf918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:35.206016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 878/1733] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:34 2022] DELETE /identity/v3/projects/c217a6065f9e4a77a573a915c372b4ea => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:35.206016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:35.206016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:35.213637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:35.247112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:35.250421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.336134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481286f-c870-4e6e-be94-f2d7880f7214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:35.336134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481286f-c870-4e6e-be94-f2d7880f7214 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:35.336134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481286f-c870-4e6e-be94-f2d7880f7214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:35.336134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481286f-c870-4e6e-be94-f2d7880f7214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:35.336578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481286f-c870-4e6e-be94-f2d7880f7214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:35.338408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481286f-c870-4e6e-be94-f2d7880f7214 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:35.350457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 856/1734] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:35.352127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:35.352520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:35.352900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:35.368045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:35.396189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 879/1735] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:35.397245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:35.408042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.409251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:35.416099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.578161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 857/1736] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:35.580028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:35.582897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.583164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfceecd7-328c-4aa1-bdbc-a345d79f454f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:35.599805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:35.600625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:35.600991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:35.607899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:35.607899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.618761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:35.620182 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b99ec-092a-4565-9d48-838d6c66acf0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:35.621500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:35.632626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:35.633242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:35.635287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:35.642217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:35.642217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:35.642420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 880/1737] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:35.642420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:35.642420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.642635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:35.666457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f637b-377f-43b7-af80-6ca1a2f70da9 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:35.670396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 858/1738] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:35.672154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:35.676798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.688194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:35.689041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:35.690589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:35.691105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] PATH_INFO: `/v3/users/aacea64ed80a4ae28eb29aaf85ac08e9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:35.704369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacea64ed80a4ae28eb29aaf85ac08e9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:35.713437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:35.730757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924748ee-35bd-4eee-924d-746b35221d3d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:35.745466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:35.746554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:35.746991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:35.747463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] PATH_INFO: `/v3/projects/9a9ddac45b4747d4b9bafcc995f0a062/users/7bee31c851394e0b91b295e32f566136/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:35.775896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9ddac45b4747d4b9bafcc995f0a062, user_id=7bee31c851394e0b91b295e32f566136, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:35.780534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:35.885158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 859/1739] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:35 2022] PUT /identity/v3/projects/9a9ddac45b4747d4b9bafcc995f0a062/users/7bee31c851394e0b91b295e32f566136/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:35.890093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:35.893660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:35.994075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacea64ed80a4ae28eb29aaf85ac08e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:36.007587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacea64ed80a4ae28eb29aaf85ac08e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:36.021358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacea64ed80a4ae28eb29aaf85ac08e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:36.027223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacea64ed80a4ae28eb29aaf85ac08e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:36.029756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3aa1d-66eb-4881-98e7-c422cee76419 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:36.043422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:36.047445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.047848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.048005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.053035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:36.076142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacea64ed80a4ae28eb29aaf85ac08e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:36.176542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 881/1740] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:35 2022] DELETE /identity/v3/users/aacea64ed80a4ae28eb29aaf85ac08e9 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:36.176542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:36.177158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:36.270506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:36.285590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:36.314735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 860/1741] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:36.319051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:36.320668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:36.346636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c76c5-6cf6-439f-90ea-d2f684375eb5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:36.352332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:36.356162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.356162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.356162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.356162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:36.357291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:36.372023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e107a3d-b2ab-4901-b60a-7e22b5605168 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:36.374495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 882/1742] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:36.385332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:36.387304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.392951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.392951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.392951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:36.392951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:36.393432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:36.396897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:36.426084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 861/1743] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:36.426084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:36.450640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c995b-7da5-4212-9575-68637fe92ad0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:36.461302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef579173-1b45-4c7c-950d-1199c12afe1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:36.466698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579173-1b45-4c7c-950d-1199c12afe1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.466698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579173-1b45-4c7c-950d-1199c12afe1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.467640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579173-1b45-4c7c-950d-1199c12afe1a admin admin] PATH_INFO: `/v3/projects/354ccf5d06b34a59a8ec96151d70f5fc/users/46e008a14e3d41b09ef9c7b8c321f9f6/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.500784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:36.500924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef579173-1b45-4c7c-950d-1199c12afe1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=354ccf5d06b34a59a8ec96151d70f5fc, user_id=46e008a14e3d41b09ef9c7b8c321f9f6, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:36.512305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef579173-1b45-4c7c-950d-1199c12afe1a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:36.644375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ec59e-ea9a-452b-b22b-d4b014de1fd4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:36.652801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:36.653822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.654114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.654671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.659801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:36.675851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:36.679943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:36.713416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 883/1744] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:36.714164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:36.725622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 862/1745] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:36 2022] PUT /identity/v3/projects/354ccf5d06b34a59a8ec96151d70f5fc/users/46e008a14e3d41b09ef9c7b8c321f9f6/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:36.741646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6557df-b7be-495d-9669-26c8cce6c5b6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.742041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6557df-b7be-495d-9669-26c8cce6c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.742493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6557df-b7be-495d-9669-26c8cce6c5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.744836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed550cdf-8537-4eb0-95aa-85182ac8e74f service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:36.850038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7a6557df-b7be-495d-9669-26c8cce6c5b6 None None] MFA Rules not processed for user `46e008a14e3d41b09ef9c7b8c321f9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:36.853766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7a6557df-b7be-495d-9669-26c8cce6c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:36.857356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f850918-4233-4d68-b7f0-79a31707b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:36.858147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f850918-4233-4d68-b7f0-79a31707b6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.858491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f850918-4233-4d68-b7f0-79a31707b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.858845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f850918-4233-4d68-b7f0-79a31707b6fd admin admin] PATH_INFO: `/v3/projects/9a9ddac45b4747d4b9bafcc995f0a062/users/7bee31c851394e0b91b295e32f566136/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.861526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f850918-4233-4d68-b7f0-79a31707b6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9ddac45b4747d4b9bafcc995f0a062, user_id=7bee31c851394e0b91b295e32f566136, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:36.866761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f850918-4233-4d68-b7f0-79a31707b6fd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:36.885163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 884/1746] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:36 2022] PUT /identity/v3/projects/9a9ddac45b4747d4b9bafcc995f0a062/users/7bee31c851394e0b91b295e32f566136/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:36.905970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:36.906851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:36.907436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:36.965798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] MFA Rules not processed for user `7bee31c851394e0b91b295e32f566136`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:36.968203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:37.036312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 863/1747] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:37.042993 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6557df-b7be-495d-9669-26c8cce6c5b6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:37.058531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:37.074841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6557df-b7be-495d-9669-26c8cce6c5b6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:37.179181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 885/1748] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:37.203514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:37.208013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:37.266276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:37.267065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:37.267586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:37.268044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:37.271642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:37.274643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:37.429204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e3b9f-4b96-4b11-b970-8c3d37d132cb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:37.442106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:37.443764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:37.449583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:37.450920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:37.466327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 864/1749] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:37 2022] POST /identity/v3/projects => generated 369 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:21:37.474166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:37.500000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:37.511773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:37.554701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 886/1750] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:37.616185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:37.618853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:37.690339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1e447-fa9d-40a4-ace9-7f91585790d1 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:37.810354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181697e9-8a1b-42c5-910f-8a46e5a64048 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:37.834593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:37.835286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:37.835576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:37.835852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:37.836531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:37.848836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:37.855516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:37.856206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:37.856499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:37.858148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] PATH_INFO: `/v3/projects/58a0e80a077f448596ec7b486a5a6937` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:37.859835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a0e80a077f448596ec7b486a5a6937)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:37.863712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:37.969904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:38.018675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 865/1751] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:37 2022] POST /identity/v3/users => generated 291 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:38.018675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:38.059500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a0e80a077f448596ec7b486a5a6937', 'request_id': 'req-cbb873e2-1fd4-4bec-af2b-044401253876'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:38.143982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944839c-b80e-4b5f-9121-382c23ff73be admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:38.148377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a0e80a077f448596ec7b486a5a6937', 'request_id': 'req-cbb873e2-1fd4-4bec-af2b-044401253876'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:38.160799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a0e80a077f448596ec7b486a5a6937', 'request_id': 'req-cbb873e2-1fd4-4bec-af2b-044401253876'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:38.199475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] The token cache is being invalidate because project 58a0e80a077f448596ec7b486a5a6937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:38.207934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:38.211870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:38.217007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 887/1752] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:37 2022] DELETE /identity/v3/projects/58a0e80a077f448596ec7b486a5a6937 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:38.218049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:38.219693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:38.239908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eb8e7-4fac-4b31-90a0-8771e94e99ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:38.240561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eb8e7-4fac-4b31-90a0-8771e94e99ad service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.240848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eb8e7-4fac-4b31-90a0-8771e94e99ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.241151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eb8e7-4fac-4b31-90a0-8771e94e99ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.244489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-928eb8e7-4fac-4b31-90a0-8771e94e99ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:38.358497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928eb8e7-4fac-4b31-90a0-8771e94e99ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:38.360402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928eb8e7-4fac-4b31-90a0-8771e94e99ad service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:38.372209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 866/1753] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:38.373909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d95e3-e1c9-4c68-97c9-b23b60431c97 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.374203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d95e3-e1c9-4c68-97c9-b23b60431c97 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.374697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d95e3-e1c9-4c68-97c9-b23b60431c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.403257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3a1d95e3-e1c9-4c68-97c9-b23b60431c97 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:38.404836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3a1d95e3-e1c9-4c68-97c9-b23b60431c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:38.490503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb873e2-1fd4-4bec-af2b-044401253876 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:38.516065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:38.516891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.517241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.517731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.518929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:38.522343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:38.561285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 888/1754] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:38.562345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:38.610788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c08c1-5cc2-40e1-bcc8-39daedd98762 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:38.619098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 867/1755] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:38.622676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:38.623442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.623729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.624014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] PATH_INFO: `/v3/projects/4cad276850b24aeeafab95233e56f36e/users/45a357096442493e971f20fc5d059d66/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.628111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3c2f-e173-4b4e-9804-e894aca20bd7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.628727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3c2f-e173-4b4e-9804-e894aca20bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.629243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3c2f-e173-4b4e-9804-e894aca20bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.661330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e3de3c2f-e173-4b4e-9804-e894aca20bd7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:38.663606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e3de3c2f-e173-4b4e-9804-e894aca20bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:38.682609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 868/1756] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:38.686676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de3c2f-e173-4b4e-9804-e894aca20bd7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:38.699040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cad276850b24aeeafab95233e56f36e, user_id=45a357096442493e971f20fc5d059d66, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:38.701742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de3c2f-e173-4b4e-9804-e894aca20bd7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:38.701903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:38.711811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:38.714341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.714882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.715434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.716455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:38.716985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:38.719384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:38.739147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 869/1757] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:21:38.744895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:38.779401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f377-125b-432c-9adb-c952e220d662 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:38.790424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:38.793791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.793791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.793791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.793791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:38.794735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 889/1758] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:38 2022] PUT /identity/v3/projects/4cad276850b24aeeafab95233e56f36e/users/45a357096442493e971f20fc5d059d66/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:38.798799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:38.798962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:38.969635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61cf8-0815-4f8d-b737-4dca626bb8a0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:38.984766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:38.985708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:38.985979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:38.986264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:38.988353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:38.992331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:39.006991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 870/1759] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:38 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:39.008709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:39.026173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 890/1760] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:39.027102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:39.046871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880f8ca-525f-4931-8cd5-db7435b39211 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:39.193073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df8faf-60d6-4eb0-9b77-060ad2fc1f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:39.196233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df8faf-60d6-4eb0-9b77-060ad2fc1f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.196892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f2f0e-14d4-4236-9195-a591febb7459 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:39.198545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df8faf-60d6-4eb0-9b77-060ad2fc1f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.213086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:39.213086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.213086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.213086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.213667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df8faf-60d6-4eb0-9b77-060ad2fc1f15 admin admin] PATH_INFO: `/v3/projects/4cad276850b24aeeafab95233e56f36e/users/45a357096442493e971f20fc5d059d66/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.213784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:39.213784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:39.213978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df8faf-60d6-4eb0-9b77-060ad2fc1f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cad276850b24aeeafab95233e56f36e, user_id=45a357096442493e971f20fc5d059d66, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:39.213978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df8faf-60d6-4eb0-9b77-060ad2fc1f15 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:39.231608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 891/1761] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:39 2022] PUT /identity/v3/projects/4cad276850b24aeeafab95233e56f36e/users/45a357096442493e971f20fc5d059d66/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:39.243487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b8956-7b64-4616-8620-bf2c1c127215 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.243720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b8956-7b64-4616-8620-bf2c1c127215 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.244100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b8956-7b64-4616-8620-bf2c1c127215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.276724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:39.312778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-500b8956-7b64-4616-8620-bf2c1c127215 None None] MFA Rules not processed for user `45a357096442493e971f20fc5d059d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:39.321454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-500b8956-7b64-4616-8620-bf2c1c127215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:39.321889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 871/1762] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:39 2022] POST /identity/v3/users => generated 289 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:39.321889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:39.339695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1f3d-1dcc-4ab6-a9e1-07f71681764f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:39.462992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:39.463757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 892/1763] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:39.464439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.464948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.465239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.466226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:39.469813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:39.495014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 872/1764] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:39.495800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b8956-7b64-4616-8620-bf2c1c127215 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:39.499857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:39.505846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:39.532264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b8956-7b64-4616-8620-bf2c1c127215 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:39.548485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:39.552998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.553542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.554018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.556402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:39.560170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:39.580069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 893/1765] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:39.584630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:39.601297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba93c7e-8145-444e-9750-8cca7a5147a0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:39.608230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:39.627515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.627810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.628113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] PATH_INFO: `/v3/projects/560780ba854d4df9a8ee83369de74a2e/users/063fbcde0b8b4d49b103b307723e49af/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.648662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d928a5-2cee-46d2-ad95-757bddbb1860 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:39.671354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=560780ba854d4df9a8ee83369de74a2e, user_id=063fbcde0b8b4d49b103b307723e49af, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:39.675151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:39.703252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad362-9b43-42bd-840e-9f4e9d20eec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:39.713025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad362-9b43-42bd-840e-9f4e9d20eec1 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.713413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad362-9b43-42bd-840e-9f4e9d20eec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.714671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad362-9b43-42bd-840e-9f4e9d20eec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.739866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efad362-9b43-42bd-840e-9f4e9d20eec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:39.744015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efad362-9b43-42bd-840e-9f4e9d20eec1 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:39.778819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 873/1766] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:39.859311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 894/1767] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:39 2022] PUT /identity/v3/projects/560780ba854d4df9a8ee83369de74a2e/users/063fbcde0b8b4d49b103b307723e49af/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:39.872416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fab4a-3033-4d9c-b7bd-c710483802b4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:39.873046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fab4a-3033-4d9c-b7bd-c710483802b4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:39.873876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fab4a-3033-4d9c-b7bd-c710483802b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:39.973920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1e3fab4a-3033-4d9c-b7bd-c710483802b4 None None] MFA Rules not processed for user `063fbcde0b8b4d49b103b307723e49af`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:39.975316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1e3fab4a-3033-4d9c-b7bd-c710483802b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:39.984538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:40.119130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a0c28-3007-431d-8ae9-3036e899fe20 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:40.125773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:40.126418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:40.126701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:40.126975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:40.132607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:40.160599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 874/1768] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:40.161949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fab4a-3033-4d9c-b7bd-c710483802b4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:40.303342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:40.303342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:40.303342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 895/1769] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:40.304435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fab4a-3033-4d9c-b7bd-c710483802b4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:40.304435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:40.304435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:40.304435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:40.305273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:40.305273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:40.305273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:40.698446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 875/1770] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:40 2022] POST /identity/v3/projects => generated 375 bytes in 537 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:40.839455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:40.839455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc6c04-daa2-45cd-b9c5-be141a670d7f service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:40.906494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:40.907259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:40.913574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:40.941516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:40.941516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:40.941516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:41.315515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 896/1771] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:40 2022] POST /identity/v3/users => generated 294 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:41.322062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:41.343882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d168a-7792-4bf1-b314-9658f0df59fd admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:41.443468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:41.443468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:41.443468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:41.443468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:41.443868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:41.443868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:41.443868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 876/1772] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:41.444063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:41.453074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7be0-5e44-4dce-a096-adee01890b77 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:41.463074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:41.463806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:41.464088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:41.464540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] PATH_INFO: `/v3/projects/bccf7f2ae40a4233916dd5ff39304736/users/108eae1a28bd4a18a95ad25965a32c3c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:41.482555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccf7f2ae40a4233916dd5ff39304736, user_id=108eae1a28bd4a18a95ad25965a32c3c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:41.485939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:42.232024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 897/1773] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:41 2022] PUT /identity/v3/projects/bccf7f2ae40a4233916dd5ff39304736/users/108eae1a28bd4a18a95ad25965a32c3c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 797 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:42.236240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:42.249990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0703ab-376f-4c2b-8a58-071ce3847214 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:42.333202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e7bb5-ce6d-4a28-9982-91177a50e027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:42.333943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e7bb5-ce6d-4a28-9982-91177a50e027 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.334036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e7bb5-ce6d-4a28-9982-91177a50e027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.334313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e7bb5-ce6d-4a28-9982-91177a50e027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.335039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e7bb5-ce6d-4a28-9982-91177a50e027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:42.336812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e7bb5-ce6d-4a28-9982-91177a50e027 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:42.350031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 877/1774] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:42.354000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:42.381900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.382283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.382661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.383549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb0e85-76dd-4dc0-9523-e364c16c7c3f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:42.395444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dd30e-71d7-404a-af0b-141b90978c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:42.396186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dd30e-71d7-404a-af0b-141b90978c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.396472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dd30e-71d7-404a-af0b-141b90978c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.396758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dd30e-71d7-404a-af0b-141b90978c6d admin admin] PATH_INFO: `/v3/projects/bccf7f2ae40a4233916dd5ff39304736/users/108eae1a28bd4a18a95ad25965a32c3c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.404650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dd30e-71d7-404a-af0b-141b90978c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccf7f2ae40a4233916dd5ff39304736, user_id=108eae1a28bd4a18a95ad25965a32c3c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:42.409959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dd30e-71d7-404a-af0b-141b90978c6d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:42.422561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 898/1775] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:42 2022] PUT /identity/v3/projects/bccf7f2ae40a4233916dd5ff39304736/users/108eae1a28bd4a18a95ad25965a32c3c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:42.425667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] MFA Rules not processed for user `14f65eca2b994097aefbfd5c80f4ba52`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:42.426933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:42.434044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a69ee7-9c61-421b-9c89-0abb6a4eef22 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.434607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a69ee7-9c61-421b-9c89-0abb6a4eef22 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.435078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a69ee7-9c61-421b-9c89-0abb6a4eef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.476446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e0a69ee7-9c61-421b-9c89-0abb6a4eef22 None None] MFA Rules not processed for user `108eae1a28bd4a18a95ad25965a32c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:42.479151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e0a69ee7-9c61-421b-9c89-0abb6a4eef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:42.521880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 878/1776] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:42.526256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:42.532169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:42.561649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 899/1777] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:42.599298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cf8b5-82a1-427f-adaf-23e97574cf84 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:42.600713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:42.601340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.601630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.601942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.602589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:42.604600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:42.732672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 879/1778] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:42 2022] POST /identity/v3/projects => generated 403 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:21:42.738138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a69ee7-9c61-421b-9c89-0abb6a4eef22 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:42.750708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a69ee7-9c61-421b-9c89-0abb6a4eef22 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:42.823831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:42.824471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.824759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.825040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.825734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:42.827808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:42.828877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:42.889400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c8c5-dceb-47e7-83e9-1fe3da6ae78c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:42.895405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:42.896383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.896508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.896873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.918059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 900/1779] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:42 2022] POST /identity/v3/users => generated 308 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:42.921516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:42.935986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f65bf-7fa6-4425-b372-82774f88cec3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:42.944546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:42.945249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:42.945542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:42.946294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:42.947161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:42.951302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:42.964991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 901/1780] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:42.969028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:42.994564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02312e-5ad9-4356-9487-05f75fe47c1a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:43.011658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:43.011658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:43.011658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:43.011658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] PATH_INFO: `/v3/projects/87d6606c42614d31bd9228ad33c5f8eb/users/a85cebc70d524e1b99fd025f629ee9e7/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:43.043031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:43.045532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:43.056542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d6606c42614d31bd9228ad33c5f8eb, user_id=a85cebc70d524e1b99fd025f629ee9e7, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:43.064027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:43.066353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 880/1781] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:21:43.158208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 902/1782] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:42 2022] PUT /identity/v3/projects/87d6606c42614d31bd9228ad33c5f8eb/users/a85cebc70d524e1b99fd025f629ee9e7/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:43.162289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:43.175036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e3ef-1aa8-4367-aa7a-acb26161f655 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:43.240924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397c91a-1d8c-465c-be6a-3dd2430fa910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:43.241570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c91a-1d8c-465c-be6a-3dd2430fa910 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:43.241857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c91a-1d8c-465c-be6a-3dd2430fa910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:43.242137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c91a-1d8c-465c-be6a-3dd2430fa910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:43.242854 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397c91a-1d8c-465c-be6a-3dd2430fa910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:43.244539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397c91a-1d8c-465c-be6a-3dd2430fa910 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:43.255061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 881/1783] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:43.259314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:43.275399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63f30f-dfda-4ae3-beb6-0723dd741a0d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:43.281940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90226fda-d3f5-4de8-8c2c-326934c6c0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:43.282995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90226fda-d3f5-4de8-8c2c-326934c6c0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:43.284010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90226fda-d3f5-4de8-8c2c-326934c6c0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:43.284331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90226fda-d3f5-4de8-8c2c-326934c6c0b5 admin admin] PATH_INFO: `/v3/projects/87d6606c42614d31bd9228ad33c5f8eb/users/a85cebc70d524e1b99fd025f629ee9e7/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:43.286385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90226fda-d3f5-4de8-8c2c-326934c6c0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d6606c42614d31bd9228ad33c5f8eb, user_id=a85cebc70d524e1b99fd025f629ee9e7, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:43.289379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90226fda-d3f5-4de8-8c2c-326934c6c0b5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:43.302399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 903/1784] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:43 2022] PUT /identity/v3/projects/87d6606c42614d31bd9228ad33c5f8eb/users/a85cebc70d524e1b99fd025f629ee9e7/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:43.811139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:44.085577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:45.056567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3504b-0af4-4c6b-a883-f8c30226a5cb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:45.057621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3504b-0af4-4c6b-a883-f8c30226a5cb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:45.059980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3504b-0af4-4c6b-a883-f8c30226a5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:45.092468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b1d3504b-0af4-4c6b-a883-f8c30226a5cb None None] MFA Rules not processed for user `a85cebc70d524e1b99fd025f629ee9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:45.094319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b1d3504b-0af4-4c6b-a883-f8c30226a5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:45.162312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 882/1785] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:46.121886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33042eac-80fc-4164-8500-0047a33fae04 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:46.122626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33042eac-80fc-4164-8500-0047a33fae04 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:46.122626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33042eac-80fc-4164-8500-0047a33fae04 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:46.124157 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 904/1786] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:21:46.159418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea51514-8c3d-45e6-9beb-34cecb6c24bb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:46.160857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea51514-8c3d-45e6-9beb-34cecb6c24bb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:46.161485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea51514-8c3d-45e6-9beb-34cecb6c24bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:46.198440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1ea51514-8c3d-45e6-9beb-34cecb6c24bb None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:46.200061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1ea51514-8c3d-45e6-9beb-34cecb6c24bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:46.304610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 883/1787] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:46.315993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33042eac-80fc-4164-8500-0047a33fae04 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:46.317722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-33042eac-80fc-4164-8500-0047a33fae04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:46.334312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33042eac-80fc-4164-8500-0047a33fae04 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:46.339375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:46.340013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:46.340297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:46.340574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:46.350980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:46.353134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:46.369055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 905/1788] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:46.657849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:47.291330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea51514-8c3d-45e6-9beb-34cecb6c24bb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:47.393007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea51514-8c3d-45e6-9beb-34cecb6c24bb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:47.398141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:47.399093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:47.399419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:47.399889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] PATH_INFO: `/v3/users/14f65eca2b994097aefbfd5c80f4ba52` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:47.401162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f65eca2b994097aefbfd5c80f4ba52)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:47.403136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:47.685729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f65eca2b994097aefbfd5c80f4ba52'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:47.686452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f65eca2b994097aefbfd5c80f4ba52'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:47.694462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f65eca2b994097aefbfd5c80f4ba52'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:47.828727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f65eca2b994097aefbfd5c80f4ba52'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:47.852144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f65eca2b994097aefbfd5c80f4ba52'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:47.864402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 884/1789] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:47 2022] DELETE /identity/v3/users/14f65eca2b994097aefbfd5c80f4ba52 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:47.931861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:48.269386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:48.298975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c409c-a248-4740-bd87-0d4da61881a1 service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:48.424790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:48.425206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:48.426102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:48.426820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] PATH_INFO: `/v3/projects/b9a08a24e6ae44f998fe04fe5dc5118a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:48.429485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a08a24e6ae44f998fe04fe5dc5118a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:48.433651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:48.491038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:48.494834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:48.512820 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba274208-907f-4985-b211-d3229630ecda admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:48.520667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:48.521406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:48.521696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:48.521976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] PATH_INFO: `/v3/users/46e008a14e3d41b09ef9c7b8c321f9f6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:48.523569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e008a14e3d41b09ef9c7b8c321f9f6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:48.525567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:48.608622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a08a24e6ae44f998fe04fe5dc5118a', 'request_id': 'req-6c79caf1-9012-4e4e-9a18-4c23ec33516c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.692244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a08a24e6ae44f998fe04fe5dc5118a', 'request_id': 'req-6c79caf1-9012-4e4e-9a18-4c23ec33516c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.703523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a08a24e6ae44f998fe04fe5dc5118a', 'request_id': 'req-6c79caf1-9012-4e4e-9a18-4c23ec33516c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.719773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] The token cache is being invalidate because project b9a08a24e6ae44f998fe04fe5dc5118a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:48.720078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.720779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.722360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 906/1790] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:48 2022] DELETE /identity/v3/projects/b9a08a24e6ae44f998fe04fe5dc5118a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:48.732188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:48.733750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:48.829976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79caf1-9012-4e4e-9a18-4c23ec33516c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:48.842116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:48.843726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:48.844266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:48.844930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] PATH_INFO: `/v3/users/a85cebc70d524e1b99fd025f629ee9e7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:48.847227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e008a14e3d41b09ef9c7b8c321f9f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.847468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a85cebc70d524e1b99fd025f629ee9e7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:48.848390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e008a14e3d41b09ef9c7b8c321f9f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.851048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:48.867374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e008a14e3d41b09ef9c7b8c321f9f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:48.867374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:49.000080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e008a14e3d41b09ef9c7b8c321f9f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.026620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cd7173a4-fa94-49af-9eb8-398a5cf73504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e008a14e3d41b09ef9c7b8c321f9f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.034809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 885/1791] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:48 2022] DELETE /identity/v3/users/46e008a14e3d41b09ef9c7b8c321f9f6 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:49.039743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:49.040138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:49.040516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:49.061598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:49.063037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:49.131734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a85cebc70d524e1b99fd025f629ee9e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.163950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a85cebc70d524e1b99fd025f629ee9e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.165480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a85cebc70d524e1b99fd025f629ee9e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.168585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a85cebc70d524e1b99fd025f629ee9e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.183830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 886/1792] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:49.205712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:49.206371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a85cebc70d524e1b99fd025f629ee9e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.210086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:49.286971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 907/1793] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:48 2022] DELETE /identity/v3/users/a85cebc70d524e1b99fd025f629ee9e7 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:21:49.326597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71df56-e425-4982-b309-0deae6ce49df None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:49.390448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e5152-aa2c-40b8-872d-94b39ac7fa8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:49.390995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e5152-aa2c-40b8-872d-94b39ac7fa8a service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:49.391506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e5152-aa2c-40b8-872d-94b39ac7fa8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:49.391833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e5152-aa2c-40b8-872d-94b39ac7fa8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:49.483129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e5152-aa2c-40b8-872d-94b39ac7fa8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:49.484896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e5152-aa2c-40b8-872d-94b39ac7fa8a service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:49.501536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 887/1794] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:49.696967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:49.696967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:49.711298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b329cc-e1be-4978-9187-53bd568eaec2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:49.716324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:49.717093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:49.717406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:49.717667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] PATH_INFO: `/v3/projects/87d6606c42614d31bd9228ad33c5f8eb` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:49.719226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d6606c42614d31bd9228ad33c5f8eb)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:49.723289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:49.727820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:49.895405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d6606c42614d31bd9228ad33c5f8eb', 'request_id': 'req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.982521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d6606c42614d31bd9228ad33c5f8eb', 'request_id': 'req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:49.990821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d6606c42614d31bd9228ad33c5f8eb', 'request_id': 'req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.002523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] The token cache is being invalidate because project 87d6606c42614d31bd9228ad33c5f8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:50.002812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.003513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.005056 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 908/1795] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:49 2022] DELETE /identity/v3/projects/87d6606c42614d31bd9228ad33c5f8eb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:50.010109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51100-d353-4b5d-931c-c81cc887cff2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.010405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51100-d353-4b5d-931c-c81cc887cff2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.010717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51100-d353-4b5d-931c-c81cc887cff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.032086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-67f51100-d353-4b5d-931c-c81cc887cff2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:50.033825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-67f51100-d353-4b5d-931c-c81cc887cff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:50.047879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.050593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:50.115762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 888/1796] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:50.124746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7626e-fb3c-4212-9055-8483e00b18c6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.125167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7626e-fb3c-4212-9055-8483e00b18c6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.125301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7626e-fb3c-4212-9055-8483e00b18c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.142406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0abb3-6b71-41cf-ad2a-8d997b246ce0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.147866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a4b7626e-fb3c-4212-9055-8483e00b18c6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:50.148093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.148884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.149178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.149394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a4b7626e-fb3c-4212-9055-8483e00b18c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:50.149516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] PATH_INFO: `/v3/projects/354ccf5d06b34a59a8ec96151d70f5fc` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.150868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=354ccf5d06b34a59a8ec96151d70f5fc)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.153027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.159631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 889/1797] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:50.164142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7626e-fb3c-4212-9055-8483e00b18c6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.174618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7626e-fb3c-4212-9055-8483e00b18c6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.180161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.180815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.181146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.181434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.182159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:50.182471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.184172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.197532 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 890/1798] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:21:50.202724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.219174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add2249-695f-4a66-b9b5-da12c09b30e5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.225260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.225998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.226474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.226721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.227682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.229773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.265774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:50.304250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354ccf5d06b34a59a8ec96151d70f5fc', 'request_id': 'req-696c2412-2be3-418f-9cb9-2d579dfa75ca'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.368614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 891/1799] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:50 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:21:50.374486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.383949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354ccf5d06b34a59a8ec96151d70f5fc', 'request_id': 'req-696c2412-2be3-418f-9cb9-2d579dfa75ca'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.395057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354ccf5d06b34a59a8ec96151d70f5fc', 'request_id': 'req-696c2412-2be3-418f-9cb9-2d579dfa75ca'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.410467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] The token cache is being invalidate because project 354ccf5d06b34a59a8ec96151d70f5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:50.410658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.411426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.413517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 909/1800] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:50 2022] DELETE /identity/v3/projects/354ccf5d06b34a59a8ec96151d70f5fc => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:50.418069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.419775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:50.495754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a9235-aa49-4803-bd32-bbdc7ccd11ae admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.506587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.508917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.509855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.510472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.511342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.514367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.520040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c2412-2be3-418f-9cb9-2d579dfa75ca admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.529236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.529236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.529236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.529236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] PATH_INFO: `/v3/users/45a357096442493e971f20fc5d059d66` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.530832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a357096442493e971f20fc5d059d66)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.532876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.622055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 892/1801] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:50 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:50.623521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.626679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:50.642100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519258c-4390-47da-8543-7d0b11a87e9b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.647778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.648462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.648755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.649036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.649780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.651515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.663329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 893/1802] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:50.666903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.677283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1c06e-2727-4900-9514-aef90c2c1fb0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.682282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.682901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.683215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.683502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.684253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.685937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.699239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 894/1803] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:50.703433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.717810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ea523-5c6c-4a5f-b0d9-a679e3fd26ed admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:50.726095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa29278-d1ea-41fa-ac29-3f731b007bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:50.726788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa29278-d1ea-41fa-ac29-3f731b007bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.727072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa29278-d1ea-41fa-ac29-3f731b007bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.727387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa29278-d1ea-41fa-ac29-3f731b007bb1 admin admin] PATH_INFO: `/v3/projects/4f4e803d323c429193a1105758bdd742/users/dea64834d796445797652c8f519b1eca/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.748271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa29278-d1ea-41fa-ac29-3f731b007bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4e803d323c429193a1105758bdd742, user_id=dea64834d796445797652c8f519b1eca, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:50.751080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa29278-d1ea-41fa-ac29-3f731b007bb1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:50.772558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a357096442493e971f20fc5d059d66'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.784427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a357096442493e971f20fc5d059d66'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.786918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a357096442493e971f20fc5d059d66'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.787548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a357096442493e971f20fc5d059d66'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.803750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a357096442493e971f20fc5d059d66'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:50.823394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 895/1804] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:50 2022] PUT /identity/v3/projects/4f4e803d323c429193a1105758bdd742/users/dea64834d796445797652c8f519b1eca/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:50.835537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055e66e-04ec-417d-971c-2d66cf8f7cf0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:50.836075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055e66e-04ec-417d-971c-2d66cf8f7cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:50.836497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055e66e-04ec-417d-971c-2d66cf8f7cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:50.866564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3055e66e-04ec-417d-971c-2d66cf8f7cf0 None None] MFA Rules not processed for user `dea64834d796445797652c8f519b1eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:50.868091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3055e66e-04ec-417d-971c-2d66cf8f7cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:50.880976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 910/1805] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:50 2022] DELETE /identity/v3/users/45a357096442493e971f20fc5d059d66 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:50.938045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 896/1806] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:50.942969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:50.946087 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:51.032002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e63fc-8bdd-4de6-ace7-e889a54fb1fb admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.037552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.038584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.039072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.039500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.040412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.042512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.108931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:51.175318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 911/1807] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:50 2022] POST /identity/v3/projects => generated 363 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:21:51.176563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:51.181738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055e66e-04ec-417d-971c-2d66cf8f7cf0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.265020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055e66e-04ec-417d-971c-2d66cf8f7cf0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.327153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.328026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.328379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.328670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.329347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.331405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.373854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.379096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:51.394731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb4142-23ca-466e-91e2-6dde0fd741d5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.401052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.401721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.402018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.402312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] PATH_INFO: `/v3/projects/4cad276850b24aeeafab95233e56f36e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.406647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cad276850b24aeeafab95233e56f36e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.406818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.446342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 897/1808] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:51 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:21:51.450497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.460719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e0c83-4a6b-4dd6-8c08-266e3af54591 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.465418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.466042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.466410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.466684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.467836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.470366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.481137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 898/1809] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:51.485660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.496049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40d4d7-4b52-47ac-a12a-2b5e5f76cae7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.500850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.501558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.501851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.502221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] PATH_INFO: `/v3/projects/caa5ff9b1b55471388fda2cdbac915dd/users/f1faf31d4cf14813980fdabf1be7587c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.514475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cad276850b24aeeafab95233e56f36e', 'request_id': 'req-4b2c10d6-3391-4b96-978b-cd5632798ef1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:51.520614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa5ff9b1b55471388fda2cdbac915dd, user_id=f1faf31d4cf14813980fdabf1be7587c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.523537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.588381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cad276850b24aeeafab95233e56f36e', 'request_id': 'req-4b2c10d6-3391-4b96-978b-cd5632798ef1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:51.597620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cad276850b24aeeafab95233e56f36e', 'request_id': 'req-4b2c10d6-3391-4b96-978b-cd5632798ef1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:51.606585 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 899/1810] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:51 2022] PUT /identity/v3/projects/caa5ff9b1b55471388fda2cdbac915dd/users/f1faf31d4cf14813980fdabf1be7587c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:21:51.610803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.612314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] The token cache is being invalidate because project 4cad276850b24aeeafab95233e56f36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:51.612784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:51.614284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b2c10d6-3391-4b96-978b-cd5632798ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:51.616414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 912/1811] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:51 2022] DELETE /identity/v3/projects/4cad276850b24aeeafab95233e56f36e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:21:51.621121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cb9f-4654-462e-86ea-dcbe0b3e70ab None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.621297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cb9f-4654-462e-86ea-dcbe0b3e70ab None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.621576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cb9f-4654-462e-86ea-dcbe0b3e70ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.642765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2da0cb9f-4654-462e-86ea-dcbe0b3e70ab None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:51.644099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2da0cb9f-4654-462e-86ea-dcbe0b3e70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:51.686567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da473727-d889-4a70-a857-e90dba08e224 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.691713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.692396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.692689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.692971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.693676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.695542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.708173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 900/1812] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:21:51.712365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.716321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:51.737443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba562-1746-4f65-9862-f4522011dc0f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.746116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94424457-6027-49d6-8d71-2bf17b73e2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.746883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94424457-6027-49d6-8d71-2bf17b73e2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.747272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94424457-6027-49d6-8d71-2bf17b73e2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.747626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94424457-6027-49d6-8d71-2bf17b73e2f2 admin admin] PATH_INFO: `/v3/projects/caa5ff9b1b55471388fda2cdbac915dd/users/f1faf31d4cf14813980fdabf1be7587c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.750541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94424457-6027-49d6-8d71-2bf17b73e2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa5ff9b1b55471388fda2cdbac915dd, user_id=f1faf31d4cf14813980fdabf1be7587c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.753569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94424457-6027-49d6-8d71-2bf17b73e2f2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.756655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 913/1813] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:51.765568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7370a-8573-454b-88cf-0b48dccc4b21 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.765861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7370a-8573-454b-88cf-0b48dccc4b21 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.766139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7370a-8573-454b-88cf-0b48dccc4b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.766239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 901/1814] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:51 2022] PUT /identity/v3/projects/caa5ff9b1b55471388fda2cdbac915dd/users/f1faf31d4cf14813980fdabf1be7587c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:21:51.776593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee25ed-6f9f-4cd3-bf21-e15a0ac539ce None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.781991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee25ed-6f9f-4cd3-bf21-e15a0ac539ce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.781991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee25ed-6f9f-4cd3-bf21-e15a0ac539ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.791462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-01a7370a-8573-454b-88cf-0b48dccc4b21 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:51.793084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-01a7370a-8573-454b-88cf-0b48dccc4b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:51.810165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b9ee25ed-6f9f-4cd3-bf21-e15a0ac539ce None None] MFA Rules not processed for user `f1faf31d4cf14813980fdabf1be7587c`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:51.811668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b9ee25ed-6f9f-4cd3-bf21-e15a0ac539ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:51.863836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:51.872854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 914/1815] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:21:51.873916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7370a-8573-454b-88cf-0b48dccc4b21 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.883951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7370a-8573-454b-88cf-0b48dccc4b21 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.888927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.889541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.889831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.890114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.890833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:51.891144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.893534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:51.898207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:51.899241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 902/1816] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:51.911298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 915/1817] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:51.914755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee25ed-6f9f-4cd3-bf21-e15a0ac539ce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.921155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:51.926262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:51.928824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee25ed-6f9f-4cd3-bf21-e15a0ac539ce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:51.935449 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:51.936195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:51.936554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:51.936893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:51.937668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:51.940252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.007254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4005-a09a-4aba-a171-33daf835973e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.011847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.012504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.012941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.013229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.025072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.026744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.043818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 916/1818] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:52.062884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 903/1819] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:51 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:52.063825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.075330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07879d63-6b15-4136-912e-4b2910922ea0 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.142572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.144065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.144359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.145316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.145695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.148433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.229460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 917/1820] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:52 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:21:52.233784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.244742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0570c7-2802-4d9a-8c17-0caa7c305d2b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.250107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.250309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.250645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.250864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.251614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.253800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.271832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 904/1821] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:52.273692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.281247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.290899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb612f35-d775-4154-8c9f-182432f9d4d2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.297062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.297983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.298416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.298841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.299851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.301860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.315430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 918/1822] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:52.321753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.339103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceca92-76ca-461f-9722-b7b43e53c098 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.348322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.349356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.349838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.350121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] PATH_INFO: `/v3/projects/7773e2ed258c475b85491ead0fdb7bf7/users/833d9fe77c44417a81c8afb05a77d0bb/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.360747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f27cc-eb83-4427-9db9-d03c40121078 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.365171 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec7d9b-ba0d-47cb-be63-c87c368a5c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.366059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec7d9b-ba0d-47cb-be63-c87c368a5c1a service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.366526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec7d9b-ba0d-47cb-be63-c87c368a5c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.366950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec7d9b-ba0d-47cb-be63-c87c368a5c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.369561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e6ec7d9b-ba0d-47cb-be63-c87c368a5c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:52.370039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7773e2ed258c475b85491ead0fdb7bf7, user_id=833d9fe77c44417a81c8afb05a77d0bb, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.374879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.434219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec7d9b-ba0d-47cb-be63-c87c368a5c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.435727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec7d9b-ba0d-47cb-be63-c87c368a5c1a service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.445695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 905/1823] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:52.503068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 919/1824] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:52 2022] PUT /identity/v3/projects/7773e2ed258c475b85491ead0fdb7bf7/users/833d9fe77c44417a81c8afb05a77d0bb/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:52.513702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.513993 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.514269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.543551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] MFA Rules not processed for user `833d9fe77c44417a81c8afb05a77d0bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:52.552738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:52.615905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 906/1825] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:52.616011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.632967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.634374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:52.689217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4227a-2473-40bb-a1a6-d87145278de3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.697150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.697788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.698241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.698697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.700041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.706826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d3156-445d-4ad9-8bcd-c67cd8bd12a9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.706950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.707035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.707035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.707035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.707035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.709277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:52.773182 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.774533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.784265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 907/1826] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:21:52.836004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 920/1827] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:52 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:52.840208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:52.851018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc005f6-559a-44b6-a4de-4176af96cc11 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:52.911741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:52.913021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:52.913306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:52.913587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:52.914259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:52.916265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:52.942756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:53.035214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:53.042896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 908/1828] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:52 2022] POST /identity/v3/users => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:53.052699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:53.063552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122839c-948b-42ff-9844-401e6bb0d7a6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:53.068426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:53.069057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:53.069337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:53.069610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:53.070311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:53.071980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:53.084366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 921/1829] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:53.088698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:53.100301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99cca7-a4a4-4302-9240-5c4d63793634 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:53.105185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:53.105881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:53.106168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:53.106448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] PATH_INFO: `/v3/projects/106b192d0b2f40a3aed26ec989387077/users/06ff2c530c2a422fbaa8c8a32884f0a9/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:53.124577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] RBAC: Authorizing `identity:create_grant(project_id=106b192d0b2f40a3aed26ec989387077, user_id=06ff2c530c2a422fbaa8c8a32884f0a9, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:53.127301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:53.252924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 909/1830] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:53 2022] PUT /identity/v3/projects/106b192d0b2f40a3aed26ec989387077/users/06ff2c530c2a422fbaa8c8a32884f0a9/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:53.261118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:53.269021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf1a85-f326-48aa-9816-83dfbce72a8d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:53.328528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ebb16-68fe-4c73-9ad1-f6f1986d6a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:53.329072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ebb16-68fe-4c73-9ad1-f6f1986d6a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:53.329355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ebb16-68fe-4c73-9ad1-f6f1986d6a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:53.329867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ebb16-68fe-4c73-9ad1-f6f1986d6a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:53.330885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ebb16-68fe-4c73-9ad1-f6f1986d6a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:53.332540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ebb16-68fe-4c73-9ad1-f6f1986d6a88 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:53.342665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 922/1831] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:53.347676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:53.358302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69755610-2441-4625-8222-79e13ebb232b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:53.363391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:53.364086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:53.364441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:53.364732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] PATH_INFO: `/v3/projects/106b192d0b2f40a3aed26ec989387077/users/06ff2c530c2a422fbaa8c8a32884f0a9/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:53.366840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=106b192d0b2f40a3aed26ec989387077, user_id=06ff2c530c2a422fbaa8c8a32884f0a9, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:53.369748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:53.381616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 910/1832] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:53 2022] PUT /identity/v3/projects/106b192d0b2f40a3aed26ec989387077/users/06ff2c530c2a422fbaa8c8a32884f0a9/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:53.393208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46310-98ae-4d53-8985-2711131f3164 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:53.393538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46310-98ae-4d53-8985-2711131f3164 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:53.393820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce46310-98ae-4d53-8985-2711131f3164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:53.425354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7ce46310-98ae-4d53-8985-2711131f3164 None None] MFA Rules not processed for user `06ff2c530c2a422fbaa8c8a32884f0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:53.426799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ce46310-98ae-4d53-8985-2711131f3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:53.504978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 923/1833] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:53.516272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:53.579589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fcfc8-72a2-47e2-ad5c-23f806e8c7b7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:53.583848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:53.593084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:53.593084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:53.593084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:53.594685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:53.596919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:53.606932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 911/1834] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:53.724508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce46310-98ae-4d53-8985-2711131f3164 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:53.740461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce46310-98ae-4d53-8985-2711131f3164 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:53.742567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb569f7-505d-403f-9fdc-de228bde0b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:53.743312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb569f7-505d-403f-9fdc-de228bde0b85 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:53.743679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb569f7-505d-403f-9fdc-de228bde0b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:53.744090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb569f7-505d-403f-9fdc-de228bde0b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:53.770841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:53.803261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb569f7-505d-403f-9fdc-de228bde0b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:53.804733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb569f7-505d-403f-9fdc-de228bde0b85 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:53.814259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 924/1835] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:54.503315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:55.160671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:55.696638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:55.730906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:56.182576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:56.421342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:56.485806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228732a-294e-4586-af3c-13d500d30085 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:56.490889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:56.492010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:56.492010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:56.492303 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] PATH_INFO: `/v3/users/dea64834d796445797652c8f519b1eca` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:56.493362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea64834d796445797652c8f519b1eca)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:56.495394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:56.510498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:56.680140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:56.765183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea64834d796445797652c8f519b1eca'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:56.766072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea64834d796445797652c8f519b1eca'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:56.776092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea64834d796445797652c8f519b1eca'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:56.877914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea64834d796445797652c8f519b1eca'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:56.891270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea64834d796445797652c8f519b1eca'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:56.897937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 912/1836] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:56 2022] DELETE /identity/v3/users/dea64834d796445797652c8f519b1eca => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:56.902666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:56.903126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:56.903262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:56.921951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:56.923222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:57.000856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 925/1837] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:57.012792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:57.014037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:57.077420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1337460-2519-4d18-98a9-8f8ce7df5451 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:57.081848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:57.082480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:57.082914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:57.083216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:57.099599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:57.101070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:57.112143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 913/1838] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:21:57.125682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:57.160029 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:57.325860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:57.596181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:57.597885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:57.613781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea6be4-93cc-48ae-864e-688300b8899e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:57.619518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:57.620311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:57.620645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:57.620996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] PATH_INFO: `/v3/projects/4f4e803d323c429193a1105758bdd742` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:57.622597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4e803d323c429193a1105758bdd742)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:57.624719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:57.733107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:57.779561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4e803d323c429193a1105758bdd742', 'request_id': 'req-8b74caef-b89d-4b84-83eb-64aa3d0ec998'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:57.810903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:57.838956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4e803d323c429193a1105758bdd742', 'request_id': 'req-8b74caef-b89d-4b84-83eb-64aa3d0ec998'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:57.845733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4e803d323c429193a1105758bdd742', 'request_id': 'req-8b74caef-b89d-4b84-83eb-64aa3d0ec998'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:57.865483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] The token cache is being invalidate because project 4f4e803d323c429193a1105758bdd742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:57.866227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:57.866991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:57.868926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 926/1839] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:57 2022] DELETE /identity/v3/projects/4f4e803d323c429193a1105758bdd742 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:57.873005 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:57.885090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef093441-d7b5-4ecd-838d-1b1c09fae650 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:57.952403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:57.953112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:57.953430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:57.953718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] PATH_INFO: `/v3/users/f1faf31d4cf14813980fdabf1be7587c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:57.954960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1faf31d4cf14813980fdabf1be7587c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:57.957539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:57.962975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:58.133140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1faf31d4cf14813980fdabf1be7587c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:58.133794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1faf31d4cf14813980fdabf1be7587c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:58.140736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1faf31d4cf14813980fdabf1be7587c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:58.239661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1faf31d4cf14813980fdabf1be7587c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:58.253985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c2835db7-df48-4d6c-96b5-5e1d1b45f749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1faf31d4cf14813980fdabf1be7587c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:58.260448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 914/1840] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:21:57 2022] DELETE /identity/v3/users/f1faf31d4cf14813980fdabf1be7587c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:58.694370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:58.696315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:58.710853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74caef-b89d-4b84-83eb-64aa3d0ec998 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:58.761897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:58.776082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:58.776658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:58.776940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:58.777218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] PATH_INFO: `/v3/projects/caa5ff9b1b55471388fda2cdbac915dd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:58.779667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa5ff9b1b55471388fda2cdbac915dd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:58.781162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:58.895478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa5ff9b1b55471388fda2cdbac915dd', 'request_id': 'req-83d9f847-b45d-49ec-9bd8-980da93b1c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:59.039618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa5ff9b1b55471388fda2cdbac915dd', 'request_id': 'req-83d9f847-b45d-49ec-9bd8-980da93b1c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:59.045712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa5ff9b1b55471388fda2cdbac915dd', 'request_id': 'req-83d9f847-b45d-49ec-9bd8-980da93b1c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:59.057296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] The token cache is being invalidate because project caa5ff9b1b55471388fda2cdbac915dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:21:59.066960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:59.066960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-83d9f847-b45d-49ec-9bd8-980da93b1c61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:21:59.066960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 927/1841] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:21:58 2022] DELETE /identity/v3/projects/caa5ff9b1b55471388fda2cdbac915dd => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:21:59.067302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982c34-3782-4222-b049-6ec17b422958 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.067738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982c34-3782-4222-b049-6ec17b422958 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.067939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982c34-3782-4222-b049-6ec17b422958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.099641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-82982c34-3782-4222-b049-6ec17b422958 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:59.100908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-82982c34-3782-4222-b049-6ec17b422958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:59.179132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 915/1842] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:59.190033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecab89b-e535-4db1-bffa-656c3bce9be9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.190327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecab89b-e535-4db1-bffa-656c3bce9be9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.190608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecab89b-e535-4db1-bffa-656c3bce9be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.218495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6ecab89b-e535-4db1-bffa-656c3bce9be9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:59.219955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6ecab89b-e535-4db1-bffa-656c3bce9be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:59.233388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 928/1843] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:59.234227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982c34-3782-4222-b049-6ec17b422958 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:59.244959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982c34-3782-4222-b049-6ec17b422958 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:59.250413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:59.251091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.251417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.251700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.252904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:21:59.253218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:59.254865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:59.267008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 916/1844] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:21:59.276816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecab89b-e535-4db1-bffa-656c3bce9be9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:59.286363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecab89b-e535-4db1-bffa-656c3bce9be9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:59.291685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:59.292476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.292912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.293195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.294059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:59.296114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:59.404144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 929/1845] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:59 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:21:59.408816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:59.420200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34f86-5d9c-4530-a36a-f79498aa8fd5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:59.482014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:59.482836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.483201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.483544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.484259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:59.487000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:59.595445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:59.600559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 917/1846] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:21:59 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:21:59.604770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:59.615760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca8d3-2c45-472e-bb9a-3cb92119e744 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:59.620196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:59.620835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.621595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.621873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.622573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:59.624210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:59.635356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 930/1847] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:59.639534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:59.649644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb53be1-7884-404d-af2b-c413c03da7b6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:59.654722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631afc20-42d3-4e7c-8f63-f088cd5add01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:59.655382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631afc20-42d3-4e7c-8f63-f088cd5add01 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.655667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631afc20-42d3-4e7c-8f63-f088cd5add01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.655944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631afc20-42d3-4e7c-8f63-f088cd5add01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.656634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631afc20-42d3-4e7c-8f63-f088cd5add01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:59.658310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631afc20-42d3-4e7c-8f63-f088cd5add01 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:59.669095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 918/1848] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:21:59.673285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:59.683903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7403461-ae89-480c-8389-74f5a8623ca4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:59.688400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:59.693393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.693393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.693393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] PATH_INFO: `/v3/projects/b4cc63ccf3ee4871af327cd035e8c575/users/9fdae50c79a94661abb79eb8cfd7ed3f/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.706538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4cc63ccf3ee4871af327cd035e8c575, user_id=9fdae50c79a94661abb79eb8cfd7ed3f, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:59.709413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:59.793886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 931/1849] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:21:59 2022] PUT /identity/v3/projects/b4cc63ccf3ee4871af327cd035e8c575/users/9fdae50c79a94661abb79eb8cfd7ed3f/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:21:59.806926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18d0c-3fcc-4371-94c1-24846cfbf1d5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.807237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18d0c-3fcc-4371-94c1-24846cfbf1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.807517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18d0c-3fcc-4371-94c1-24846cfbf1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.834467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-47b18d0c-3fcc-4371-94c1-24846cfbf1d5 None None] MFA Rules not processed for user `9fdae50c79a94661abb79eb8cfd7ed3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:21:59.835694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-47b18d0c-3fcc-4371-94c1-24846cfbf1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:21:59.903755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 919/1850] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:21:59.908983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:59.976837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c10c-bab4-4616-a563-b6ddfba247a6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:59.981916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:59.982712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:59.983098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:59.983517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:59.984304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:59.987251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:00.113145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 932/1851] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:21:59 2022] POST /identity/v3/projects => generated 369 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:00.114717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b18d0c-3fcc-4371-94c1-24846cfbf1d5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:00.125456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b18d0c-3fcc-4371-94c1-24846cfbf1d5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:00.181911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:00.182542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:00.182825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:00.183103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:00.183832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:00.185758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:00.285881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 920/1852] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:00 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:00.289882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:00.295441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:00.301329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addd669-407d-41cd-9e22-932faf8b1e9e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:00.308440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:00.310178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:00.310178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:00.310178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:00.310496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:00.312145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:00.324509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 933/1853] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:00.328950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:00.339288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb407b-2193-43c9-af8d-cf053eaf5b78 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:00.343973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:00.344799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:00.345081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:00.345362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] PATH_INFO: `/v3/projects/c2d583f716d744f09b205f01a431fc4c/users/f0233bc2246c4def9771225b6ec623cc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:00.364818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d583f716d744f09b205f01a431fc4c, user_id=f0233bc2246c4def9771225b6ec623cc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:00.367545 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:00.520716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 921/1854] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:00 2022] PUT /identity/v3/projects/c2d583f716d744f09b205f01a431fc4c/users/f0233bc2246c4def9771225b6ec623cc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:00.525085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:00.536682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd9986-37e5-4229-9329-4fa67141072a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:00.594939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2367e7-f628-43a0-a2c4-0c724021b7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:00.595621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2367e7-f628-43a0-a2c4-0c724021b7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:00.596432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2367e7-f628-43a0-a2c4-0c724021b7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:00.596714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2367e7-f628-43a0-a2c4-0c724021b7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:00.597418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2367e7-f628-43a0-a2c4-0c724021b7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:00.599104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2367e7-f628-43a0-a2c4-0c724021b7ad admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:00.609464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 934/1855] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:00.614956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:00.625477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f7f4e-6264-42f7-97a7-be902de94e8c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:00.630391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:00.631002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:00.631327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:00.631655 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] PATH_INFO: `/v3/projects/c2d583f716d744f09b205f01a431fc4c/users/f0233bc2246c4def9771225b6ec623cc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:00.633597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d583f716d744f09b205f01a431fc4c, user_id=f0233bc2246c4def9771225b6ec623cc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:00.636446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:00.647702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 922/1856] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:00 2022] PUT /identity/v3/projects/c2d583f716d744f09b205f01a431fc4c/users/f0233bc2246c4def9771225b6ec623cc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:00.658811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1cb0b-c495-4686-9a01-8dc05fef15c1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:00.659110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1cb0b-c495-4686-9a01-8dc05fef15c1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:00.659426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1cb0b-c495-4686-9a01-8dc05fef15c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:00.688584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d1c1cb0b-c495-4686-9a01-8dc05fef15c1 None None] MFA Rules not processed for user `f0233bc2246c4def9771225b6ec623cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:00.689850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d1c1cb0b-c495-4686-9a01-8dc05fef15c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:00.758738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 935/1857] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:00.773183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:00.774848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:00.811451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:00.836433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea1179-04c9-497b-a48f-c7f1fe5a4537 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:00.840286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626090c0-7763-42ac-b182-4992d8a7846e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:00.840934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626090c0-7763-42ac-b182-4992d8a7846e service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:00.841195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626090c0-7763-42ac-b182-4992d8a7846e service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:00.841468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626090c0-7763-42ac-b182-4992d8a7846e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:00.852267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626090c0-7763-42ac-b182-4992d8a7846e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:00.853794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626090c0-7763-42ac-b182-4992d8a7846e service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:00.864014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 923/1858] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:01.390959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeb806-20ea-423f-9cb5-82f33fa92c41 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:01.391178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeb806-20ea-423f-9cb5-82f33fa92c41 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:01.391673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeb806-20ea-423f-9cb5-82f33fa92c41 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:01.393756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 936/1859] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:22:01.403705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:01.404232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:01.404800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:01.425422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:01.426819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:01.509850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 924/1860] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:22:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:01.524754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeb806-20ea-423f-9cb5-82f33fa92c41 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:01.526420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9eeb806-20ea-423f-9cb5-82f33fa92c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:01.548388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeb806-20ea-423f-9cb5-82f33fa92c41 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:01.554154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd7829-d9ef-412f-bca8-90e3d224cc77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:01.554777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7829-d9ef-412f-bca8-90e3d224cc77 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:01.555061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7829-d9ef-412f-bca8-90e3d224cc77 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:01.555372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7829-d9ef-412f-bca8-90e3d224cc77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:01.565805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd7829-d9ef-412f-bca8-90e3d224cc77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:01.567448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd7829-d9ef-412f-bca8-90e3d224cc77 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:01.578996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 937/1861] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:04.113883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:04.115738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:04.288259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3114b04-e7ac-4aca-9636-8f83799a845e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:04.296911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:04.297598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:04.297890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:04.298174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] PATH_INFO: `/v3/users/833d9fe77c44417a81c8afb05a77d0bb` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:04.301158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=833d9fe77c44417a81c8afb05a77d0bb)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:04.306519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:04.535952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833d9fe77c44417a81c8afb05a77d0bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:04.536870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833d9fe77c44417a81c8afb05a77d0bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:04.547067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833d9fe77c44417a81c8afb05a77d0bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:04.628365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833d9fe77c44417a81c8afb05a77d0bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:04.634384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:04.642936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833d9fe77c44417a81c8afb05a77d0bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:04.649987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 925/1862] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:04 2022] DELETE /identity/v3/users/833d9fe77c44417a81c8afb05a77d0bb => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:04.654754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:04.655336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:04.655425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:04.679941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:04.681376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:04.751066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:04.761215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 938/1863] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:04.774023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:04.775770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:04.846125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478847e-5945-4e07-aaa2-fa6905d37ed9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:04.849811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:04.850429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:04.850828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:04.851084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:04.860804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:04.863043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:04.880238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 926/1864] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:05.247752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:05.249646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:05.263695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09c0b0-e0ce-4325-a7ef-1708ae7e97e3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:05.269903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:05.271035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:05.278488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:05.278488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] PATH_INFO: `/v3/users/9fdae50c79a94661abb79eb8cfd7ed3f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:05.278488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fdae50c79a94661abb79eb8cfd7ed3f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:05.278488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:05.595343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:05.598745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:05.616322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9c922-aa29-4624-a89c-c2b1ffa80909 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:05.622760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:05.623473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:05.624339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:05.624339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] PATH_INFO: `/v3/projects/7773e2ed258c475b85491ead0fdb7bf7` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:05.625456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7773e2ed258c475b85491ead0fdb7bf7)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:05.627512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:05.648742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fdae50c79a94661abb79eb8cfd7ed3f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.660411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fdae50c79a94661abb79eb8cfd7ed3f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.661270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fdae50c79a94661abb79eb8cfd7ed3f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.664577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fdae50c79a94661abb79eb8cfd7ed3f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.675247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc59d036-1e95-472f-8ee5-5cf1c8e3d08d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fdae50c79a94661abb79eb8cfd7ed3f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.746468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7773e2ed258c475b85491ead0fdb7bf7', 'request_id': 'req-96de0468-9230-4801-bf71-afc1395cdbff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.753847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7773e2ed258c475b85491ead0fdb7bf7', 'request_id': 'req-96de0468-9230-4801-bf71-afc1395cdbff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.760892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 939/1865] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:05 2022] DELETE /identity/v3/users/9fdae50c79a94661abb79eb8cfd7ed3f => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:05.767306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:05.768538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:05.768829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:05.788711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:05.789964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:05.851119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7773e2ed258c475b85491ead0fdb7bf7', 'request_id': 'req-96de0468-9230-4801-bf71-afc1395cdbff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.856297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 940/1866] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:05.867061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] The token cache is being invalidate because project 7773e2ed258c475b85491ead0fdb7bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:05.867389 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.868371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:05.868543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:05.869947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 927/1867] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:05 2022] DELETE /identity/v3/projects/7773e2ed258c475b85491ead0fdb7bf7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:05.871164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:05.874041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:05.875892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:05.941602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea072a0-dc9c-4132-bf38-8437c5ffa0a9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:05.946785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:05.947546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:05.947897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:05.948256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:05.951113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:05.957995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de0468-9230-4801-bf71-afc1395cdbff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:05.963759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:05.969733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:05.969733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:05.969733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] PATH_INFO: `/v3/users/06ff2c530c2a422fbaa8c8a32884f0a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:05.971785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ff2c530c2a422fbaa8c8a32884f0a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:05.973311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:05.974947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:05.976467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:05.997817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 941/1868] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:06.011960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:06.230883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ff2c530c2a422fbaa8c8a32884f0a9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.231646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ff2c530c2a422fbaa8c8a32884f0a9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.244101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ff2c530c2a422fbaa8c8a32884f0a9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.371094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ff2c530c2a422fbaa8c8a32884f0a9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.385091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ff2c530c2a422fbaa8c8a32884f0a9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.392896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 928/1869] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:05 2022] DELETE /identity/v3/users/06ff2c530c2a422fbaa8c8a32884f0a9 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:06.475581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:06.475581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:06.485196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e4856-b1b8-419d-8e28-cecc37bef312 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:06.528144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:06.598067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:06.598732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:06.599097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:06.599549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] PATH_INFO: `/v3/projects/b4cc63ccf3ee4871af327cd035e8c575` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:06.600803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4cc63ccf3ee4871af327cd035e8c575)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:06.603316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:06.788700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4cc63ccf3ee4871af327cd035e8c575', 'request_id': 'req-f676a6b6-8210-4ea1-a037-a1054380a39b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.863498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4cc63ccf3ee4871af327cd035e8c575', 'request_id': 'req-f676a6b6-8210-4ea1-a037-a1054380a39b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.870439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4cc63ccf3ee4871af327cd035e8c575', 'request_id': 'req-f676a6b6-8210-4ea1-a037-a1054380a39b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.884836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] The token cache is being invalidate because project b4cc63ccf3ee4871af327cd035e8c575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:06.885146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.885817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:06.890926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 942/1870] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:06 2022] DELETE /identity/v3/projects/b4cc63ccf3ee4871af327cd035e8c575 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:06.892301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:06.894120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:06.909943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44615d0a-3665-482f-a7e1-a01c23842c31 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:06.976875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:06.977594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:06.977853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:06.978128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] PATH_INFO: `/v3/users/f0233bc2246c4def9771225b6ec623cc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:06.979515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0233bc2246c4def9771225b6ec623cc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:06.981811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:07.024598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:07.026203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:07.052610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a6b6-8210-4ea1-a037-a1054380a39b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:07.066631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:07.069599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:07.069599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:07.069599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] PATH_INFO: `/v3/projects/106b192d0b2f40a3aed26ec989387077` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:07.089490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=106b192d0b2f40a3aed26ec989387077)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:07.091928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:07.227715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0233bc2246c4def9771225b6ec623cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.228521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0233bc2246c4def9771225b6ec623cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.230258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106b192d0b2f40a3aed26ec989387077', 'request_id': 'req-3b223f1a-dc77-49eb-b320-a9ff5d190909'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.244218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0233bc2246c4def9771225b6ec623cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.360290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106b192d0b2f40a3aed26ec989387077', 'request_id': 'req-3b223f1a-dc77-49eb-b320-a9ff5d190909'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.378256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106b192d0b2f40a3aed26ec989387077', 'request_id': 'req-3b223f1a-dc77-49eb-b320-a9ff5d190909'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.393463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] The token cache is being invalidate because project 106b192d0b2f40a3aed26ec989387077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:07.393790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.394583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3b223f1a-dc77-49eb-b320-a9ff5d190909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.396613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 943/1871] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:07 2022] DELETE /identity/v3/projects/106b192d0b2f40a3aed26ec989387077 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:07.401332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd23d8b-f28a-4df6-99d0-e6dea23da291 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:07.401690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd23d8b-f28a-4df6-99d0-e6dea23da291 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:07.402065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd23d8b-f28a-4df6-99d0-e6dea23da291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:07.422961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8dd23d8b-f28a-4df6-99d0-e6dea23da291 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:07.424212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8dd23d8b-f28a-4df6-99d0-e6dea23da291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:07.461342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:07.498863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 944/1872] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:07.524046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4da6-9377-4dd7-947e-e8738fe01c84 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:07.524339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4da6-9377-4dd7-947e-e8738fe01c84 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:07.524623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4da6-9377-4dd7-947e-e8738fe01c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:07.543462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0233bc2246c4def9771225b6ec623cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.565107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ee9a4da6-9377-4dd7-947e-e8738fe01c84 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:07.566662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ee9a4da6-9377-4dd7-947e-e8738fe01c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:07.568178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0233bc2246c4def9771225b6ec623cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:07.577457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 945/1873] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:07.579953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 929/1874] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:06 2022] DELETE /identity/v3/users/f0233bc2246c4def9771225b6ec623cc => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:07.584898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4da6-9377-4dd7-947e-e8738fe01c84 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:07.603554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4da6-9377-4dd7-947e-e8738fe01c84 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:07.726234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:07.726875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:07.727163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:07.727476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:07.728185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:07.728497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:07.730189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:07.743164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 946/1875] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:07.748181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:07.749909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:07.767117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff87ee-dc6d-4362-b88c-567490683f61 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:07.772709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:07.773357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:07.773635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:07.773922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:07.774707 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:07.776646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:07.783397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:07.935929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 930/1876] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:07 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:07.940661 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:07.956568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13f3f-b662-4211-b288-3a25ffd3a37b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.043491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.044622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.045083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.046164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.047176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.050990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.152544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 947/1877] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:07 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:08.156292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.167052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015777a5-f4c1-4659-8959-fa83c50a5a3b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.171946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.172958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.173413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.173768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.174541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.176254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.187013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 931/1878] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:08.191622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.203424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6b119-1009-4d06-b4e5-2157258787ae admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.212892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.213544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.213965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.214193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.214980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.217080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.236506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 948/1879] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:08.242407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.256089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02376db1-2a45-44bb-b8d7-8358bb2109e0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.279812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcaa27-1f1a-4f76-aa4d-e7b079a513dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.280520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcaa27-1f1a-4f76-aa4d-e7b079a513dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.280915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcaa27-1f1a-4f76-aa4d-e7b079a513dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.281315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcaa27-1f1a-4f76-aa4d-e7b079a513dc admin admin] PATH_INFO: `/v3/projects/6ae88cfb9a96454db829e1c199a7083d/users/fe1dcd9fc7854a0c9cc2b641ae837993/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.314986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcaa27-1f1a-4f76-aa4d-e7b079a513dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae88cfb9a96454db829e1c199a7083d, user_id=fe1dcd9fc7854a0c9cc2b641ae837993, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.314986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcaa27-1f1a-4f76-aa4d-e7b079a513dc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.322980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.325015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:08.362954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff312a2-4366-4ce3-b4d9-31421c0fe67e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.374941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.375529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.375819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.376232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] PATH_INFO: `/v3/projects/c2d583f716d744f09b205f01a431fc4c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.378033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d583f716d744f09b205f01a431fc4c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.380073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.398567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 932/1880] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:08 2022] PUT /identity/v3/projects/6ae88cfb9a96454db829e1c199a7083d/users/fe1dcd9fc7854a0c9cc2b641ae837993/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:08.409126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374aea3-2a8f-4f3c-8f43-9485a5dd3b8f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.409386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374aea3-2a8f-4f3c-8f43-9485a5dd3b8f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.409668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374aea3-2a8f-4f3c-8f43-9485a5dd3b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.447892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e374aea3-2a8f-4f3c-8f43-9485a5dd3b8f None None] MFA Rules not processed for user `fe1dcd9fc7854a0c9cc2b641ae837993`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:08.449220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e374aea3-2a8f-4f3c-8f43-9485a5dd3b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:08.520802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 933/1881] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:08.526243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374aea3-2a8f-4f3c-8f43-9485a5dd3b8f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.561158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:08.577116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d583f716d744f09b205f01a431fc4c', 'request_id': 'req-94564286-6905-4afb-8940-cec420434ae7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:08.602767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374aea3-2a8f-4f3c-8f43-9485a5dd3b8f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.659199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.659829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.660119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.660398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.661064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.663010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.667888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d583f716d744f09b205f01a431fc4c', 'request_id': 'req-94564286-6905-4afb-8940-cec420434ae7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:08.675485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d583f716d744f09b205f01a431fc4c', 'request_id': 'req-94564286-6905-4afb-8940-cec420434ae7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:08.688100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] The token cache is being invalidate because project c2d583f716d744f09b205f01a431fc4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:08.688400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:08.689119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-94564286-6905-4afb-8940-cec420434ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:08.691387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 949/1882] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:08 2022] DELETE /identity/v3/projects/c2d583f716d744f09b205f01a431fc4c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:08.695932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988551e-baee-4d29-9f42-2482b35410d2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.696208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988551e-baee-4d29-9f42-2482b35410d2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.696496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988551e-baee-4d29-9f42-2482b35410d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.700558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:08.717171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6988551e-baee-4d29-9f42-2482b35410d2 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:08.718451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6988551e-baee-4d29-9f42-2482b35410d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:08.727112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 950/1883] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:08.737199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c11e-fc23-483c-9e99-827991ee9fc0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.737486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c11e-fc23-483c-9e99-827991ee9fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.737763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c11e-fc23-483c-9e99-827991ee9fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.756443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0946c11e-fc23-483c-9e99-827991ee9fc0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:08.764425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0946c11e-fc23-483c-9e99-827991ee9fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:08.765445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 951/1884] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:08.769483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c11e-fc23-483c-9e99-827991ee9fc0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.778257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c11e-fc23-483c-9e99-827991ee9fc0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.782909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.784071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.784357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.784633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.785321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:08.785628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.787226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.790434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 934/1885] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:08 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:08.795798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.797829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:08.803014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 952/1886] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:22:08.803973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:08.817225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23fa60-6741-47ef-9ac9-3c4841493378 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.875624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0edb4-5db4-4150-be11-b94b86b403e1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:08.883004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.883866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.884229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.884701 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.885447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.887508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:08.887648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:08.888644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:08.889033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:08.889399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:08.890076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:08.892094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.008234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 935/1887] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:08 2022] POST /identity/v3/users => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:09.017336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.017504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 953/1888] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:08 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:09.018468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.034709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96a3f7-95ae-4c3a-9c4f-f98cf04fddad admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.099466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.100143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.100840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.101270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.102748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.104764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.106571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30bddb-ed19-4eb4-942b-ddd64c62bb66 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.112860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.113707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.114144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.114437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.115084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.117043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.118407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 936/1889] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:09.118407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.118407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:09.135218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db587193-d89f-4067-9181-0cb9ff18f4b8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.140505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.141181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.141465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.141736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] PATH_INFO: `/v3/users/b8f6775d15e9479bb25ea9f83346872d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.143518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8f6775d15e9479bb25ea9f83346872d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.145440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.220159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 954/1890] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:09 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:09.220159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.228602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39cd6-e1f7-4b13-91db-e3e7e38332d7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.233319 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.233970 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.234259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.234544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] PATH_INFO: `/v3/projects/f47591a3c22e4a9d9680949e280433b6/users/864dcff438b145cead338f2426f53981/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.239653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:09.251558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47591a3c22e4a9d9680949e280433b6, user_id=864dcff438b145cead338f2426f53981, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.254275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.333558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 955/1891] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:09 2022] PUT /identity/v3/projects/f47591a3c22e4a9d9680949e280433b6/users/864dcff438b145cead338f2426f53981/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:09.334821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.342436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8f6775d15e9479bb25ea9f83346872d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:09.345113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8f6775d15e9479bb25ea9f83346872d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:09.349856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:09.351972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8f6775d15e9479bb25ea9f83346872d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:09.417508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e8a66-5d50-4c58-906c-9eb6f24fb1f7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.422511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.423148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.423484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.423777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.424222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8f6775d15e9479bb25ea9f83346872d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:09.424481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.426235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.437181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 956/1892] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:09.438052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.440527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:09.442989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8f6775d15e9479bb25ea9f83346872d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:09.453107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 937/1893] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:09 2022] DELETE /identity/v3/users/b8f6775d15e9479bb25ea9f83346872d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:09.453107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.454039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:09.456174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae10c-f653-4673-a4ea-d2c7d069242e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.544875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ec1e-9795-4858-a255-beb548a1c936 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.549813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.550641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.550748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b63c00-21a4-47fe-8219-7beb3468bd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.550949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.551229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.551428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63c00-21a4-47fe-8219-7beb3468bd7e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.551714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63c00-21a4-47fe-8219-7beb3468bd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.551999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63c00-21a4-47fe-8219-7beb3468bd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.552706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b63c00-21a4-47fe-8219-7beb3468bd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.552902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.554424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b63c00-21a4-47fe-8219-7beb3468bd7e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.554650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.565346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 957/1894] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:09.566659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b66dae-dc0c-428b-89c2-7719352371aa None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.566783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 938/1895] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:09.567244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b66dae-dc0c-428b-89c2-7719352371aa None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.567244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b66dae-dc0c-428b-89c2-7719352371aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.570918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.586847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637f6d2-c12b-4aae-896c-450519b742aa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.588376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-01b66dae-dc0c-428b-89c2-7719352371aa None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:09.589577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-01b66dae-dc0c-428b-89c2-7719352371aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:09.594921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.594921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.594921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.594921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] PATH_INFO: `/v3/projects/f47591a3c22e4a9d9680949e280433b6/users/864dcff438b145cead338f2426f53981/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.595501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47591a3c22e4a9d9680949e280433b6, user_id=864dcff438b145cead338f2426f53981, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.598189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 958/1896] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:09.598333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.599373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b66dae-dc0c-428b-89c2-7719352371aa None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.612531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b66dae-dc0c-428b-89c2-7719352371aa None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.623070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 939/1897] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:09 2022] PUT /identity/v3/projects/f47591a3c22e4a9d9680949e280433b6/users/864dcff438b145cead338f2426f53981/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:09.623070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:09.642225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:09.716038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefe56-2afa-48ef-8879-373923fbea3e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:09.720686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8696-bc41-4b35-8f7e-35542050d209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.727050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8696-bc41-4b35-8f7e-35542050d209 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.727050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8696-bc41-4b35-8f7e-35542050d209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.727050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8696-bc41-4b35-8f7e-35542050d209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.744476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc924ae-bf02-40d2-aa68-063eab5c299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:09.745533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc924ae-bf02-40d2-aa68-063eab5c299a admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.745828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc924ae-bf02-40d2-aa68-063eab5c299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.746115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc924ae-bf02-40d2-aa68-063eab5c299a admin admin] PATH_INFO: `/v3/projects/4f97385925364e55a2ee558553835af3/users/b057f678f43242739b6379a43d13f44a/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.761372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:09.764260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc924ae-bf02-40d2-aa68-063eab5c299a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f97385925364e55a2ee558553835af3, user_id=b057f678f43242739b6379a43d13f44a, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.767067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc924ae-bf02-40d2-aa68-063eab5c299a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.802293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8696-bc41-4b35-8f7e-35542050d209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:09.803880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8696-bc41-4b35-8f7e-35542050d209 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:09.814861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 940/1898] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:09.824816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d45d-339d-4a2c-9299-bf0bb013c35c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.824816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d45d-339d-4a2c-9299-bf0bb013c35c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.830882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d45d-339d-4a2c-9299-bf0bb013c35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.882296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-aac1d45d-339d-4a2c-9299-bf0bb013c35c None None] MFA Rules not processed for user `864dcff438b145cead338f2426f53981`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:09.883941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aac1d45d-339d-4a2c-9299-bf0bb013c35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:09.887666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:09.893609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 959/1899] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:09 2022] PUT /identity/v3/projects/4f97385925364e55a2ee558553835af3/users/b057f678f43242739b6379a43d13f44a/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:09.904431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:09.905093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:09.905567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:09.941427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] MFA Rules not processed for user `b057f678f43242739b6379a43d13f44a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:09.942831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:09.989612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 941/1900] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:10.031049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 960/1901] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:10.036323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1d45d-339d-4a2c-9299-bf0bb013c35c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.046625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1d45d-339d-4a2c-9299-bf0bb013c35c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.106877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.107830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.108394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.108917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.109762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.112353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.281024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 942/1902] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:10 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:10.285371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.286064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.290400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:10.388471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638657a8-7b86-42f5-b451-ec7a0c125ddc admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.394908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.395919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.396291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.396681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.397570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.400274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.407258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3ee98-749b-490d-bb4b-f61528b970e6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.468055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.471447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.471447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.471447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.528699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 943/1903] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:10 2022] POST /identity/v3/users => generated 300 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:10.528699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.537473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:10.557957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014dec13-4a6d-4098-813a-b74f97909c9b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.563248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.563974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.564786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.565074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] PATH_INFO: `/v3/projects/07b7f2d0b8744522b90de1dd4946a977` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.566908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b7f2d0b8744522b90de1dd4946a977)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.568982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.579221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.581954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.593281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 961/1904] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:10.594269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.608360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9b79e-d940-4250-a77b-6a2034bbafdb service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.614188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.614817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.615100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.615413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.616116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.617724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.629416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 962/1905] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:10.633875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.645239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe5310-b686-4747-be3d-a3f037fa0cce admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.650709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.651426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.651720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.652002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] PATH_INFO: `/v3/projects/634a4276528d4ee39745cc992d52fab2/users/ddd60d39a9cb4cfda4a6c2716065898c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.679582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=634a4276528d4ee39745cc992d52fab2, user_id=ddd60d39a9cb4cfda4a6c2716065898c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.683433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.692767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b7f2d0b8744522b90de1dd4946a977', 'request_id': 'req-8bf952b7-3615-4fa6-89d0-7f8f1899516e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:10.699897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b7f2d0b8744522b90de1dd4946a977', 'request_id': 'req-8bf952b7-3615-4fa6-89d0-7f8f1899516e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:10.776586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b7f2d0b8744522b90de1dd4946a977', 'request_id': 'req-8bf952b7-3615-4fa6-89d0-7f8f1899516e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:10.784667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 963/1906] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:10 2022] PUT /identity/v3/projects/634a4276528d4ee39745cc992d52fab2/users/ddd60d39a9cb4cfda4a6c2716065898c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:10.788332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.790503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] The token cache is being invalidate because project 07b7f2d0b8744522b90de1dd4946a977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:10.790920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:10.791900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:10.793802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 944/1907] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:10 2022] DELETE /identity/v3/projects/07b7f2d0b8744522b90de1dd4946a977 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:10.797651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.799707 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:10.836790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:10.865991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87e68c-b6e9-4df4-b7e6-19cabf3cebc8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.877581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.878291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.878578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.878857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.883427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.884817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.891773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf952b7-3615-4fa6-89d0-7f8f1899516e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.898115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.899004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.899994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.899994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] PATH_INFO: `/v3/users/7bee31c851394e0b91b295e32f566136` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.901772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bee31c851394e0b91b295e32f566136)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:10.902635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 964/1908] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:10.905125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:10.907310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:10.907485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:10.977232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62728e2e-16b1-40b2-a892-6e861434383d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:10.981921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:10.982719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:10.983006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:10.983323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:10.985336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:11.079562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:11.086025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:11.093481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bee31c851394e0b91b295e32f566136'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:11.094421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bee31c851394e0b91b295e32f566136'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:11.105363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 965/1909] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:11.106933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:11.108358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bee31c851394e0b91b295e32f566136'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:11.110397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:11.142814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217214a8-5a22-43f3-ae49-3f3618f3602d service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:11.152898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe6385-d178-41bd-883e-464eaedcf87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:11.153863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe6385-d178-41bd-883e-464eaedcf87a admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:11.154324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe6385-d178-41bd-883e-464eaedcf87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:11.154797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe6385-d178-41bd-883e-464eaedcf87a admin admin] PATH_INFO: `/v3/projects/634a4276528d4ee39745cc992d52fab2/users/ddd60d39a9cb4cfda4a6c2716065898c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:11.157491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe6385-d178-41bd-883e-464eaedcf87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=634a4276528d4ee39745cc992d52fab2, user_id=ddd60d39a9cb4cfda4a6c2716065898c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:11.162272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe6385-d178-41bd-883e-464eaedcf87a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:11.179371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 966/1910] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:11 2022] PUT /identity/v3/projects/634a4276528d4ee39745cc992d52fab2/users/ddd60d39a9cb4cfda4a6c2716065898c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:11.191145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:11.191709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:11.192381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:11.210012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bee31c851394e0b91b295e32f566136'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:11.231865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bee31c851394e0b91b295e32f566136'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:11.239582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] MFA Rules not processed for user `ddd60d39a9cb4cfda4a6c2716065898c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:11.241449 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:11.242360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 945/1911] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:10 2022] DELETE /identity/v3/users/7bee31c851394e0b91b295e32f566136 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:11.343453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:11.371128 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 967/1912] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:11.505843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:11.508411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:11.600589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53687fb4-c9fa-45c2-a410-8990cd3d9f9e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:11.604860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:11.605518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:11.605804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:11.606086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:11.617279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:11.618812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:11.630881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 946/1913] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:11.676664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:11.676664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:11.773911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6757-388e-4d54-8e5c-7d34757d38e7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:11.778640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:11.781059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:11.781059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:11.781059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] PATH_INFO: `/v3/users/b057f678f43242739b6379a43d13f44a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:11.781327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b057f678f43242739b6379a43d13f44a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:11.783974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:11.814594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:11.816640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:11.845160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd17d6-8f40-4b45-a1dd-3af8a8e571ee service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:11.851899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:11.852625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:11.852920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:11.857844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] PATH_INFO: `/v3/projects/9a9ddac45b4747d4b9bafcc995f0a062` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:11.857844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9ddac45b4747d4b9bafcc995f0a062)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:11.857844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:11.915769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:12.029813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9ddac45b4747d4b9bafcc995f0a062', 'request_id': 'req-9c8def3a-f332-4c2b-96e0-528e5ce9768f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.039345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9ddac45b4747d4b9bafcc995f0a062', 'request_id': 'req-9c8def3a-f332-4c2b-96e0-528e5ce9768f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.049880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b057f678f43242739b6379a43d13f44a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.061724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b057f678f43242739b6379a43d13f44a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.062818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b057f678f43242739b6379a43d13f44a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.063702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b057f678f43242739b6379a43d13f44a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.079296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-221e24f1-2c04-46c6-8ec9-c229b1071bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b057f678f43242739b6379a43d13f44a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.136617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9ddac45b4747d4b9bafcc995f0a062', 'request_id': 'req-9c8def3a-f332-4c2b-96e0-528e5ce9768f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.149109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] The token cache is being invalidate because project 9a9ddac45b4747d4b9bafcc995f0a062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:12.149531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.150476 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.155380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 947/1914] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:11 2022] DELETE /identity/v3/projects/9a9ddac45b4747d4b9bafcc995f0a062 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:12.155380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:12.155380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:12.231544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8def3a-f332-4c2b-96e0-528e5ce9768f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:12.237226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:12.238118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:12.238599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:12.239033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] PATH_INFO: `/v3/users/fe1dcd9fc7854a0c9cc2b641ae837993` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:12.240302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1dcd9fc7854a0c9cc2b641ae837993)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:12.242275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:12.324388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 968/1915] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:11 2022] DELETE /identity/v3/users/b057f678f43242739b6379a43d13f44a => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:12.324388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b26b4-5969-4216-aed6-8697984d8f61 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:12.325256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b26b4-5969-4216-aed6-8697984d8f61 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:12.325579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b26b4-5969-4216-aed6-8697984d8f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:12.364322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0d1b26b4-5969-4216-aed6-8697984d8f61 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:12.364518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0d1b26b4-5969-4216-aed6-8697984d8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:12.432132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 969/1916] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:12.433424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029975f7-8e9a-4cae-8eaa-5b80018bdb35 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:12.433768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029975f7-8e9a-4cae-8eaa-5b80018bdb35 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:12.434080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029975f7-8e9a-4cae-8eaa-5b80018bdb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:12.461475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-029975f7-8e9a-4cae-8eaa-5b80018bdb35 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:12.463239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-029975f7-8e9a-4cae-8eaa-5b80018bdb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:12.474595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 970/1917] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:12.474923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:12.475214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:12.475519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:12.759615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:12.759615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:12.759615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 971/1918] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:12.759615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:12.759615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:12.762264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff0836-3cf0-425d-85ba-4a811a71175c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:12.762264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:12.762264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:12.762264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:12.762757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:12.762757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:12.762757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:12.762757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 972/1919] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:12.762757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:12.762757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075e927-6f9c-4578-b112-2fd5556802b4 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:12.763416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:12.763416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:12.763416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:12.763416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:12.763837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:12.763837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:12.763837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:12.763837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 973/1920] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:12.763837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:12.773543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e29f5d-e296-407e-95aa-1ff657bc4a42 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:12.778413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:12.779111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:12.779468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:12.779750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:12.780545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:12.782478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:12.833345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1dcd9fc7854a0c9cc2b641ae837993'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.833345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1dcd9fc7854a0c9cc2b641ae837993'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:12.833345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1dcd9fc7854a0c9cc2b641ae837993'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.164221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1dcd9fc7854a0c9cc2b641ae837993'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.177184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c71587d0-1e3b-486c-8153-04fff8d548a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1dcd9fc7854a0c9cc2b641ae837993'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.184857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 974/1921] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:12 2022] POST /identity/v3/projects => generated 381 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:13.185616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:13.188616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 948/1922] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:12 2022] DELETE /identity/v3/users/fe1dcd9fc7854a0c9cc2b641ae837993 => generated 0 bytes in 1036 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:13.192798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:13.206277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.206277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.206277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.231893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:13.233570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:13.320404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b9e3e-760e-41f5-b49c-da42d703968c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:13.345772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 949/1923] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:13.358484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:13.361404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:13.363101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:13.466164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cfc66-97b9-477c-99f8-915c2df6a5c6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:13.466388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:13.466872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.467084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.467382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.468059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:13.469985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:13.470107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:13.470739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.471022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.471401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.489552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:13.491114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:13.504278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 950/1924] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:13.588991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:13.590695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:13.605501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b34ac-d8c6-4f24-aed9-686d876c605a service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:13.614269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:13.615634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.616628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.616628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] PATH_INFO: `/v3/projects/4f97385925364e55a2ee558553835af3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.618284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f97385925364e55a2ee558553835af3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:13.621098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:13.626312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 975/1925] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:13 2022] POST /identity/v3/users => generated 291 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:13.631489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:13.642349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015473d6-2353-47ec-b119-d4e39a9068dc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:13.648356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:13.649635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.649635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.649635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.650385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:13.652317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:13.672863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 976/1926] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:13.675094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:13.687944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93121-6853-429e-8dc2-4c21ab0a3a3d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:13.699931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:13.700654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.700999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.701281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.702022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:13.703676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:13.723448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 977/1927] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:13.724529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:13.736967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc6ba1-c641-48c6-b3b3-29ac7e7b2eba admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:13.743691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608ea9b-f869-4c6b-a4fa-e1a081043551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:13.744382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608ea9b-f869-4c6b-a4fa-e1a081043551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.744664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608ea9b-f869-4c6b-a4fa-e1a081043551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.744939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608ea9b-f869-4c6b-a4fa-e1a081043551 admin admin] PATH_INFO: `/v3/projects/5b714fb6cd524ffb94745ca2181139de/users/e7e3af6f11dd4428a99b7a1aa7c76097/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.774710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608ea9b-f869-4c6b-a4fa-e1a081043551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b714fb6cd524ffb94745ca2181139de, user_id=e7e3af6f11dd4428a99b7a1aa7c76097, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:13.778170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608ea9b-f869-4c6b-a4fa-e1a081043551 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:13.788540 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f97385925364e55a2ee558553835af3', 'request_id': 'req-82719f6c-2180-407f-896b-32f0042afcbb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.799474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f97385925364e55a2ee558553835af3', 'request_id': 'req-82719f6c-2180-407f-896b-32f0042afcbb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.948212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f97385925364e55a2ee558553835af3', 'request_id': 'req-82719f6c-2180-407f-896b-32f0042afcbb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.952280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:13.964289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 978/1928] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:13 2022] PUT /identity/v3/projects/5b714fb6cd524ffb94745ca2181139de/users/e7e3af6f11dd4428a99b7a1aa7c76097/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:13.972644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] The token cache is being invalidate because project 4f97385925364e55a2ee558553835af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:13.973197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.974086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:13.974848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:13.975316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:13.976476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:13.976731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 951/1929] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:13 2022] DELETE /identity/v3/projects/4f97385925364e55a2ee558553835af3 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:13.980416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:13.982387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:14.010915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] MFA Rules not processed for user `e7e3af6f11dd4428a99b7a1aa7c76097`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:14.012467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:14.078343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82719f6c-2180-407f-896b-32f0042afcbb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:14.108228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 979/1930] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:14.112871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:14.114516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:14.187312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:14.188111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:14.188474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:14.188821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] PATH_INFO: `/v3/users/ddd60d39a9cb4cfda4a6c2716065898c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:14.191349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddd60d39a9cb4cfda4a6c2716065898c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:14.194756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:14.202236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115577b-cc5d-49c9-aeab-5191b80573e8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:14.208973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:14.209787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:14.210152 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:14.210433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:14.211392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:14.213900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:14.448271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 980/1931] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:14 2022] POST /identity/v3/projects => generated 379 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:14.449282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:14.504685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddd60d39a9cb4cfda4a6c2716065898c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:14.505510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddd60d39a9cb4cfda4a6c2716065898c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:14.514069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddd60d39a9cb4cfda4a6c2716065898c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:14.544275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d22471-6f8c-4a4f-999e-e9494825d9c9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:14.550339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:14.550985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:14.551301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:14.551578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:14.552321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:14.554683 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:14.702318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddd60d39a9cb4cfda4a6c2716065898c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:14.716245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddd60d39a9cb4cfda4a6c2716065898c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:14.729863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 952/1932] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:13 2022] DELETE /identity/v3/users/ddd60d39a9cb4cfda4a6c2716065898c => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:14.729863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:14.729863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:14.764463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 981/1933] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:14 2022] POST /identity/v3/users => generated 296 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:14.772832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:14.775166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:14.788928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565168bc-2e6e-4110-a7b9-4c5afac70fec admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:14.820232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351fc29-a872-4f5e-b3bb-d2e2416566aa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:14.825880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:14.826716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:14.827075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:14.827474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] PATH_INFO: `/v3/projects/6ae88cfb9a96454db829e1c199a7083d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:14.828750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae88cfb9a96454db829e1c199a7083d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:14.830898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:14.873267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:14.874585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:14.875048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:14.875536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:14.876265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:14.878009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:14.889302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 982/1934] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:14.893297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:14.910921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31c2d-0b7c-4287-b7b8-6f20dcaa8190 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:14.913699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:14.914377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:14.914664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:14.914946 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] PATH_INFO: `/v3/projects/2861d36a33474873a5bf8614e44cada9/users/244904a197c54c66ba4c4c970753c81c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:14.934215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2861d36a33474873a5bf8614e44cada9, user_id=244904a197c54c66ba4c4c970753c81c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:14.938584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:14.955887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae88cfb9a96454db829e1c199a7083d', 'request_id': 'req-a7b9cf62-8959-40f7-8728-c7ff836e157c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:14.962839 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae88cfb9a96454db829e1c199a7083d', 'request_id': 'req-a7b9cf62-8959-40f7-8728-c7ff836e157c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.030866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 983/1935] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:14 2022] PUT /identity/v3/projects/2861d36a33474873a5bf8614e44cada9/users/244904a197c54c66ba4c4c970753c81c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:15.035219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:15.081535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae88cfb9a96454db829e1c199a7083d', 'request_id': 'req-a7b9cf62-8959-40f7-8728-c7ff836e157c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.094987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] The token cache is being invalidate because project 6ae88cfb9a96454db829e1c199a7083d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:15.096014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.096724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.101304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 953/1936] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:14 2022] DELETE /identity/v3/projects/6ae88cfb9a96454db829e1c199a7083d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:15.102520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:15.104003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:15.111634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33900e6-0065-49af-a107-ee61c8615cea admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:15.116753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:15.117398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.117939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.118241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.118981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:15.120670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:15.136083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 984/1937] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:15.138843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:15.140814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:15.155035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c60757-2b3d-4c1f-934f-eacc6eb919c8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:15.160444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:15.161355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.161707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.162000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] PATH_INFO: `/v3/projects/2861d36a33474873a5bf8614e44cada9/users/244904a197c54c66ba4c4c970753c81c/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.168280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2861d36a33474873a5bf8614e44cada9, user_id=244904a197c54c66ba4c4c970753c81c, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:15.174916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:15.186753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cf62-8959-40f7-8728-c7ff836e157c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:15.200188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:15.200867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.204573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 985/1938] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:15 2022] PUT /identity/v3/projects/2861d36a33474873a5bf8614e44cada9/users/244904a197c54c66ba4c4c970753c81c/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:15.205310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.205605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] PATH_INFO: `/v3/users/864dcff438b145cead338f2426f53981` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.205719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:15.212780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=864dcff438b145cead338f2426f53981)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:15.214865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:15.217589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:15.303638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ba48d-3022-4197-96c4-a5dbae25a30d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:15.308247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:15.308893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.309177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.309455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] PATH_INFO: `/v3/projects/634a4276528d4ee39745cc992d52fab2` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.310615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=634a4276528d4ee39745cc992d52fab2)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:15.312698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:15.479994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '864dcff438b145cead338f2426f53981'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.480663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '864dcff438b145cead338f2426f53981'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.483620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634a4276528d4ee39745cc992d52fab2', 'request_id': 'req-538b58a7-1c88-419c-8836-27f2aa880a32'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.491073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '864dcff438b145cead338f2426f53981'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.599491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634a4276528d4ee39745cc992d52fab2', 'request_id': 'req-538b58a7-1c88-419c-8836-27f2aa880a32'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.605841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634a4276528d4ee39745cc992d52fab2', 'request_id': 'req-538b58a7-1c88-419c-8836-27f2aa880a32'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.618434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] The token cache is being invalidate because project 634a4276528d4ee39745cc992d52fab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:15.618726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.619455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-538b58a7-1c88-419c-8836-27f2aa880a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.620987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 986/1939] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:15 2022] DELETE /identity/v3/projects/634a4276528d4ee39745cc992d52fab2 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:15.622181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.622464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.622739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.649554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] MFA Rules not processed for user `244904a197c54c66ba4c4c970753c81c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:15.650748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:15.685277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '864dcff438b145cead338f2426f53981'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.698070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3e7f184a-c700-42cb-afe9-b4d53c58d2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '864dcff438b145cead338f2426f53981'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:15.704454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 954/1940] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:15 2022] DELETE /identity/v3/users/864dcff438b145cead338f2426f53981 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:15.708516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5104a-524d-4c0c-ad8f-d2066d68ac5c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.708807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5104a-524d-4c0c-ad8f-d2066d68ac5c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.709083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5104a-524d-4c0c-ad8f-d2066d68ac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.728053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 987/1941] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:15.732446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f2a5104a-524d-4c0c-ad8f-d2066d68ac5c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:15.732446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f2a5104a-524d-4c0c-ad8f-d2066d68ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:15.735540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:15.737897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:15.800855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82b296-49ee-4d30-abbe-0c8001ba2f93 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:15.803273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 955/1942] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:15.811396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:15.813711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9979d2-5c48-46a4-9b88-f4bcc8b42608 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.814013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9979d2-5c48-46a4-9b88-f4bcc8b42608 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.814482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9979d2-5c48-46a4-9b88-f4bcc8b42608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.837457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6a9979d2-5c48-46a4-9b88-f4bcc8b42608 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:15.838828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6a9979d2-5c48-46a4-9b88-f4bcc8b42608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:15.847813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 956/1943] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:15.852149 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9979d2-5c48-46a4-9b88-f4bcc8b42608 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:15.857211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:15.857843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.858125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.858399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.862111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9979d2-5c48-46a4-9b88-f4bcc8b42608 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:15.866822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:15.867537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.867899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.868251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.869030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:15.869415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:15.871118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:15.882788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 957/1944] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:22:15.887059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:15.897393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f18c76-305e-4621-9bb7-b6a8902b906a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:15.902129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:15.902743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:15.903033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:15.903351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:15.904148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:15.906073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:15.927449 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:15.930026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:15.941456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 988/1945] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:16.009759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 958/1946] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:15 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:16.013922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.032261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076fbf5-03d7-45ea-8934-1ddcff8002c1 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.100101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.100753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.101080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.101358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.102058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.104110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.224509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 989/1947] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:16 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:16.228827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.247055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcc7ea-2fc1-486c-88fc-b61b6427d896 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.252523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.253630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.254214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.254558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.254941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.255867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.257804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.261200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:16.271376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 959/1948] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:16.278666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.280640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3957a0a-77a2-4694-bc65-a466d82b178b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.285862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.286527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.286818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.287100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] PATH_INFO: `/v3/projects/f47591a3c22e4a9d9680949e280433b6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.289338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47591a3c22e4a9d9680949e280433b6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.290417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abfc28-b189-40af-8b56-2a43f0070470 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.291412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.295767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.296397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.296687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.296969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.297675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.299490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.313794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 960/1949] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:16.315717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.326397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85455c-1d70-4126-9dff-0f032b82392a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.333084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd93064-c6d3-4d1d-8c61-eb3439f3442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.333294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd93064-c6d3-4d1d-8c61-eb3439f3442b admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.333453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd93064-c6d3-4d1d-8c61-eb3439f3442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.333685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd93064-c6d3-4d1d-8c61-eb3439f3442b admin admin] PATH_INFO: `/v3/projects/2562f743952349e4a4bb310e0ede0e0a/users/19164ea012924460a1508132dd8aee49/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.349365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd93064-c6d3-4d1d-8c61-eb3439f3442b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2562f743952349e4a4bb310e0ede0e0a, user_id=19164ea012924460a1508132dd8aee49, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.352613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd93064-c6d3-4d1d-8c61-eb3439f3442b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.365852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:16.539536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 961/1950] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:16 2022] PUT /identity/v3/projects/2562f743952349e4a4bb310e0ede0e0a/users/19164ea012924460a1508132dd8aee49/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:16.547847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47591a3c22e4a9d9680949e280433b6', 'request_id': 'req-5654a870-a78f-46f6-8fcf-0175b83d361a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:16.550685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4b414-ffd1-4f39-b34f-43ce44ff33d9 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.550983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4b414-ffd1-4f39-b34f-43ce44ff33d9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.551302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4b414-ffd1-4f39-b34f-43ce44ff33d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.587604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f0b4b414-ffd1-4f39-b34f-43ce44ff33d9 None None] MFA Rules not processed for user `19164ea012924460a1508132dd8aee49`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:16.588897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f0b4b414-ffd1-4f39-b34f-43ce44ff33d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:16.657276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47591a3c22e4a9d9680949e280433b6', 'request_id': 'req-5654a870-a78f-46f6-8fcf-0175b83d361a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:16.665461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 962/1951] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:16.668987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47591a3c22e4a9d9680949e280433b6', 'request_id': 'req-5654a870-a78f-46f6-8fcf-0175b83d361a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:16.673325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4b414-ffd1-4f39-b34f-43ce44ff33d9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.676825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] The token cache is being invalidate because project f47591a3c22e4a9d9680949e280433b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:16.677122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:16.677789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5654a870-a78f-46f6-8fcf-0175b83d361a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:16.679458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 990/1952] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:16 2022] DELETE /identity/v3/projects/f47591a3c22e4a9d9680949e280433b6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:16.684486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c771dc0-7018-4e42-8ea4-d87c12ea2f01 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.684773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c771dc0-7018-4e42-8ea4-d87c12ea2f01 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.685086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c771dc0-7018-4e42-8ea4-d87c12ea2f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.706100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5c771dc0-7018-4e42-8ea4-d87c12ea2f01 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:16.707365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5c771dc0-7018-4e42-8ea4-d87c12ea2f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:16.734529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4b414-ffd1-4f39-b34f-43ce44ff33d9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.739579 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf319f96-dbcc-4c11-a1cc-94079811c019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.740217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf319f96-dbcc-4c11-a1cc-94079811c019 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.740538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf319f96-dbcc-4c11-a1cc-94079811c019 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.740827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf319f96-dbcc-4c11-a1cc-94079811c019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.742778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cf319f96-dbcc-4c11-a1cc-94079811c019 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:16.755270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf319f96-dbcc-4c11-a1cc-94079811c019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.756791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf319f96-dbcc-4c11-a1cc-94079811c019 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.767155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 963/1953] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:16.774640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 991/1954] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:16.782766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af88c10-fe49-4660-8d1b-43a5bd1c7853 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.783062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af88c10-fe49-4660-8d1b-43a5bd1c7853 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.783383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af88c10-fe49-4660-8d1b-43a5bd1c7853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.804710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7af88c10-fe49-4660-8d1b-43a5bd1c7853 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:16.805943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7af88c10-fe49-4660-8d1b-43a5bd1c7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:16.817291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 964/1955] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:16.821753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c771dc0-7018-4e42-8ea4-d87c12ea2f01 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.844432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c771dc0-7018-4e42-8ea4-d87c12ea2f01 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.851782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.852764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.853224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.853684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.854833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:16.855395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.857904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.879960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 992/1956] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:16.884643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af88c10-fe49-4660-8d1b-43a5bd1c7853 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:16.897387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af88c10-fe49-4660-8d1b-43a5bd1c7853 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:16.899691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:16.900314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:16.900603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:16.900874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:16.901558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:16.903482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:16.999035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 965/1957] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:16 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:17.000857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.011096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb0be-6ff3-4e19-b5c1-6f8c10fe4d75 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.070151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.070693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.070978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.071283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.071988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.074306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.167743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 993/1958] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:16 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:17.176673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.190779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac45b7-7591-4ae1-ad43-843f8b38e830 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.195888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.201456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.201456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.201456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.201456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.201456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.207722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.210245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:17.217418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 966/1959] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:17.222230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.234342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a11923-4c8d-47ee-a52b-2c62b9bbb333 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.240340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.241777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.241777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.241777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.242196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.243859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.257077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 967/1960] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:17.261630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.276466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb15b4d-9956-4cff-809a-98fa70690e7b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.278566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95173ef2-2e2f-425e-93a4-127c83a8e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.279516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95173ef2-2e2f-425e-93a4-127c83a8e1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.279992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95173ef2-2e2f-425e-93a4-127c83a8e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.280780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95173ef2-2e2f-425e-93a4-127c83a8e1bc admin admin] PATH_INFO: `/v3/projects/d1915bea3ac94b3ba6529eb23ea52c2e/users/9ae724b52eb24159b88595317e3d5aa1/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.291891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042bec1-da81-41ff-9e3b-ec9045cd3965 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.297677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244904a197c54c66ba4c4c970753c81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2861d36a33474873a5bf8614e44cada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.298458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.299516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.300168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.301197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:17.305034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.305169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95173ef2-2e2f-425e-93a4-127c83a8e1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1915bea3ac94b3ba6529eb23ea52c2e, user_id=9ae724b52eb24159b88595317e3d5aa1, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.305265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.305340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95173ef2-2e2f-425e-93a4-127c83a8e1bc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.316026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 994/1961] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:22:17.320406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.331222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fe7da-c222-4c22-b1cb-46d9a22a864a tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.336111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244904a197c54c66ba4c4c970753c81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2861d36a33474873a5bf8614e44cada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.336745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.337037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.337320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.338001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.339965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.395724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 968/1962] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:17 2022] PUT /identity/v3/projects/d1915bea3ac94b3ba6529eb23ea52c2e/users/9ae724b52eb24159b88595317e3d5aa1/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:17.408789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.408789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.408789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.437859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] MFA Rules not processed for user `9ae724b52eb24159b88595317e3d5aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:17.439103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:17.481532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 995/1963] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:17 2022] POST /identity/v3/projects => generated 353 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:17.486287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.514826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 969/1964] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:17.529201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.530696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:17.561906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5ded-f4ea-4e2f-8b90-72ca767cc4bb tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.570492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244904a197c54c66ba4c4c970753c81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2861d36a33474873a5bf8614e44cada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.571097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.571454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.571752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.572481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:17.572782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.574681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.592132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 996/1965] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:22:17.700536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:17.714061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fd98c-f8f9-4cc7-9d8b-b41de8bdfa0e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.717331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b739426-57c1-420f-9481-1715ad67a739 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:17.722976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '244904a197c54c66ba4c4c970753c81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2861d36a33474873a5bf8614e44cada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.723757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.724279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.724620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] PATH_INFO: `/v3/projects/d0b634e22ff24bcca10232e7d4e4fa8e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.726148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d0b634e22ff24bcca10232e7d4e4fa8e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.728564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.771674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:17.772301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:17.772583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:17.772865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:17.829760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:17.831914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:17.843410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 970/1966] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:17.867205 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:17.887428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b634e22ff24bcca10232e7d4e4fa8e', 'request_id': 'req-c527cc84-e303-40b1-8866-bff289f2caf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:18.005439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b634e22ff24bcca10232e7d4e4fa8e', 'request_id': 'req-c527cc84-e303-40b1-8866-bff289f2caf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:18.011931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b634e22ff24bcca10232e7d4e4fa8e', 'request_id': 'req-c527cc84-e303-40b1-8866-bff289f2caf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:18.025703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] The token cache is being invalidate because project d0b634e22ff24bcca10232e7d4e4fa8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:18.026014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:18.027014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c527cc84-e303-40b1-8866-bff289f2caf6 tempest-VolumeQuotaClassesTest-655226223 tempest-VolumeQuotaClassesTest-655226223-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:18.029417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 997/1967] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:17 2022] DELETE /identity/v3/projects/d0b634e22ff24bcca10232e7d4e4fa8e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:18.043965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:19.125588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:19.270419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:19.271912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:19.345678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fd1ad-abc5-448e-baa1-5646b098de01 service glance] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:19.403969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:19.404623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:19.404908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:19.405184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] PATH_INFO: `/v3/users/e7e3af6f11dd4428a99b7a1aa7c76097` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:19.406346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e3af6f11dd4428a99b7a1aa7c76097)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:19.408360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:19.468337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:19.778432 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e3af6f11dd4428a99b7a1aa7c76097'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:19.779132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e3af6f11dd4428a99b7a1aa7c76097'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:19.786637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e3af6f11dd4428a99b7a1aa7c76097'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:19.863039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e3af6f11dd4428a99b7a1aa7c76097'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:19.878630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e3af6f11dd4428a99b7a1aa7c76097'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:19.889454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 971/1968] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:19 2022] DELETE /identity/v3/users/e7e3af6f11dd4428a99b7a1aa7c76097 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:19.894050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:19.894344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:19.894693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:19.913665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:19.914922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:19.984850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 998/1969] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:20.005577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:20.007071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:20.069076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87903f91-a5ef-4d57-99e9-ffe4e24e6140 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:20.073036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:20.073673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:20.073988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:20.074226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:20.092812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:20.094559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:20.107633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 972/1970] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:20.457612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:20.847687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:20.849432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:20.874922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c727-2322-43c7-97b6-2bf79b56f9c8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:20.883045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:20.883045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:20.883429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:20.883753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] PATH_INFO: `/v3/projects/5b714fb6cd524ffb94745ca2181139de` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:20.885027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b714fb6cd524ffb94745ca2181139de)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:20.902893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:21.089411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b714fb6cd524ffb94745ca2181139de', 'request_id': 'req-566ab102-aa22-46fc-b2a5-4174853ec52f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.096878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:21.188117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b714fb6cd524ffb94745ca2181139de', 'request_id': 'req-566ab102-aa22-46fc-b2a5-4174853ec52f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.198325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b714fb6cd524ffb94745ca2181139de', 'request_id': 'req-566ab102-aa22-46fc-b2a5-4174853ec52f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.215220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] The token cache is being invalidate because project 5b714fb6cd524ffb94745ca2181139de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:21.215519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.216710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.218466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 999/1971] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:20 2022] DELETE /identity/v3/projects/5b714fb6cd524ffb94745ca2181139de => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:21.223508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:21.234206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45544daa-f394-48b0-9f7a-f1efaf244eb1 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:21.305249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:21.305908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:21.306317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:21.306597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] PATH_INFO: `/v3/users/244904a197c54c66ba4c4c970753c81c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:21.307833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] RBAC: Authorizing `identity:delete_user(user_id=244904a197c54c66ba4c4c970753c81c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:21.309774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:21.724794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '244904a197c54c66ba4c4c970753c81c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.725859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '244904a197c54c66ba4c4c970753c81c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.739944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '244904a197c54c66ba4c4c970753c81c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.833827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '244904a197c54c66ba4c4c970753c81c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.847779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-704419c8-ed2f-423d-a0c1-bc0671de13df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '244904a197c54c66ba4c4c970753c81c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:21.854512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 973/1972] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:21 2022] DELETE /identity/v3/users/244904a197c54c66ba4c4c970753c81c => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:21.982491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:22.326955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:22.328453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:22.343472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:22.347960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ab102-aa22-46fc-b2a5-4174853ec52f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:22.422079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:22.422649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:22.422934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:22.423239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] PATH_INFO: `/v3/projects/2861d36a33474873a5bf8614e44cada9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:22.424469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2861d36a33474873a5bf8614e44cada9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:22.426407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:22.542575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:22.544149 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2861d36a33474873a5bf8614e44cada9', 'request_id': 'req-bc45bc1b-eee4-419b-b00d-db781e4dc150'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:22.628628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2861d36a33474873a5bf8614e44cada9', 'request_id': 'req-bc45bc1b-eee4-419b-b00d-db781e4dc150'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:22.635533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2861d36a33474873a5bf8614e44cada9', 'request_id': 'req-bc45bc1b-eee4-419b-b00d-db781e4dc150'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:22.648598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] The token cache is being invalidate because project 2861d36a33474873a5bf8614e44cada9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:22.648897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:22.649635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc45bc1b-eee4-419b-b00d-db781e4dc150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:22.651209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1000/1973] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:22 2022] DELETE /identity/v3/projects/2861d36a33474873a5bf8614e44cada9 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:22.656641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000b689-3fd1-4934-a01c-4b1f9ae9eb44 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:22.656939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000b689-3fd1-4934-a01c-4b1f9ae9eb44 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:22.657226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000b689-3fd1-4934-a01c-4b1f9ae9eb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:22.680732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3000b689-3fd1-4934-a01c-4b1f9ae9eb44 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:22.680732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3000b689-3fd1-4934-a01c-4b1f9ae9eb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:22.739848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 974/1974] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:22.749035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c252-9ae8-4ca0-a165-36cf71b3c9a9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:22.749320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c252-9ae8-4ca0-a165-36cf71b3c9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:22.749666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c252-9ae8-4ca0-a165-36cf71b3c9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:22.769951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0ab0c252-9ae8-4ca0-a165-36cf71b3c9a9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:22.771171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0ab0c252-9ae8-4ca0-a165-36cf71b3c9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:22.779150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1001/1975] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:22.783581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000b689-3fd1-4934-a01c-4b1f9ae9eb44 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:22.793178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000b689-3fd1-4934-a01c-4b1f9ae9eb44 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:22.798030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:22.798936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:22.801327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:22.801327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:22.801327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:22.801327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:22.802169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:22.815733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 975/1976] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:22.820485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c252-9ae8-4ca0-a165-36cf71b3c9a9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:22.830839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c252-9ae8-4ca0-a165-36cf71b3c9a9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:22.835873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:22.836525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:22.836810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:22.837089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:22.837794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:22.839826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:22.945478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1002/1977] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:22 2022] POST /identity/v3/projects => generated 407 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:22.949985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:22.965219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec204d2-c57c-4ea1-9d53-3a79f779eae6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.025712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.026464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.026746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.027044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.027806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.029787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:23.039995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:23.074221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:23.142769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 976/1978] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:22 2022] POST /identity/v3/users => generated 304 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:23.147053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:23.158554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10c375-b9bc-46b8-9544-b9096c872b80 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.163221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.163932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.164220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.164503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.165205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.166817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:23.178315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1003/1979] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:23.183657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:23.193898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63eb97-f102-496b-a57e-e11ac731b81b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.199330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba809f-9050-47cd-86c1-c3c7181add7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.200208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba809f-9050-47cd-86c1-c3c7181add7f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.200512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba809f-9050-47cd-86c1-c3c7181add7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.201176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba809f-9050-47cd-86c1-c3c7181add7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.201945 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba809f-9050-47cd-86c1-c3c7181add7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.203805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba809f-9050-47cd-86c1-c3c7181add7f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:23.215646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 977/1980] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:23.224271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:23.235052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da15631-5e88-4246-b629-8f491ae8f39d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.240568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:23.242351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.243037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.254960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.254960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] PATH_INFO: `/v3/projects/e893d04bd50d40bd9c69f1b624dfe2c9/users/00968a5e985e4b29861816170b870464/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.259701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e893d04bd50d40bd9c69f1b624dfe2c9, user_id=00968a5e985e4b29861816170b870464, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.262397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:23.320242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:23.394832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1004/1981] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:23 2022] PUT /identity/v3/projects/e893d04bd50d40bd9c69f1b624dfe2c9/users/00968a5e985e4b29861816170b870464/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:23.405169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2f87c-dd79-4e6d-b280-437b9a9e7ee6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.405457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2f87c-dd79-4e6d-b280-437b9a9e7ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.405797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2f87c-dd79-4e6d-b280-437b9a9e7ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.435866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-26b2f87c-dd79-4e6d-b280-437b9a9e7ee6 None None] MFA Rules not processed for user `00968a5e985e4b29861816170b870464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:23.438346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-26b2f87c-dd79-4e6d-b280-437b9a9e7ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:23.501423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 978/1982] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:23.505962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:23.513923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:23.569819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17298cf7-6328-496b-a3ed-40942e6a1b7a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.574507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.575135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.575453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.575752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.576418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.578322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:23.616753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:23.681178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1005/1983] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:23 2022] POST /identity/v3/projects => generated 409 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:23.685285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2f87c-dd79-4e6d-b280-437b9a9e7ee6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:23.696976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2f87c-dd79-4e6d-b280-437b9a9e7ee6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.756845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.757382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.757670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.757948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.758616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.760627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:23.799679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:23.868481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 979/1984] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:23 2022] POST /identity/v3/users => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:23.872535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:23.883890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c192-6911-481f-87cc-14edc0d93c36 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.888953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.889286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.889601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.889887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.890588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.892893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:23.903984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1006/1985] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:23.908266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:23.918858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b781e-1076-4cb1-abc8-582b5a9d3cdf admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:23.924315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:23.925035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:23.925320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:23.925596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] PATH_INFO: `/v3/projects/e8c6508f19414c39a4b22525dc3a0360/users/5d2e5ab58d634a6f90d3761cbb32ca11/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:23.943202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c6508f19414c39a4b22525dc3a0360, user_id=5d2e5ab58d634a6f90d3761cbb32ca11, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:23.946071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:24.017164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 980/1986] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:23 2022] PUT /identity/v3/projects/e8c6508f19414c39a4b22525dc3a0360/users/5d2e5ab58d634a6f90d3761cbb32ca11/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:24.027485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:24.037723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a856e0-bdd6-4415-af80-fb502820e3f5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:24.073931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:24.099867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa624d-dbc0-41ac-93c9-7879cab74149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:24.100519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa624d-dbc0-41ac-93c9-7879cab74149 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:24.101440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa624d-dbc0-41ac-93c9-7879cab74149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:24.101440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa624d-dbc0-41ac-93c9-7879cab74149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:24.102207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa624d-dbc0-41ac-93c9-7879cab74149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:24.104284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa624d-dbc0-41ac-93c9-7879cab74149 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:24.117625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1007/1987] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:24.122844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:24.134554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b02f3-aff9-4b77-a585-567f9b871fb1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:24.139789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:24.140459 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:24.140744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:24.141023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] PATH_INFO: `/v3/projects/e8c6508f19414c39a4b22525dc3a0360/users/5d2e5ab58d634a6f90d3761cbb32ca11/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:24.142962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c6508f19414c39a4b22525dc3a0360, user_id=5d2e5ab58d634a6f90d3761cbb32ca11, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:24.145645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:24.156213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 981/1988] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:24 2022] PUT /identity/v3/projects/e8c6508f19414c39a4b22525dc3a0360/users/5d2e5ab58d634a6f90d3761cbb32ca11/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:24.167429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:24.167717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:24.167993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:24.196229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] MFA Rules not processed for user `5d2e5ab58d634a6f90d3761cbb32ca11`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:24.197442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:24.271013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1008/1989] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:24 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:24.281886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:24.283607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:24.342685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259979e-5995-47ff-9442-741b8143c548 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:24.346537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:24.347168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:24.347497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:24.347764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:24.357208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:24.358675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:24.368906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 982/1990] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:24 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:24.905002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:24.906507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:24.919883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e50c0-e2e4-4aca-b5c0-bcb5e010f54e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:24.930569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07694d80-e7d8-4ad7-9938-b275036a9217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:24.930569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07694d80-e7d8-4ad7-9938-b275036a9217 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:24.930569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07694d80-e7d8-4ad7-9938-b275036a9217 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:24.930569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07694d80-e7d8-4ad7-9938-b275036a9217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:24.975144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:24.999331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07694d80-e7d8-4ad7-9938-b275036a9217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:25.000988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07694d80-e7d8-4ad7-9938-b275036a9217 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:25.010810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1009/1991] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:24 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:28.606394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:28.607932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:28.675206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e58bae-bda7-4d58-9c7b-3f1d33d0e804 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:28.683549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:28.683549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:28.683549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:28.683549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] PATH_INFO: `/v3/users/968296b8db55464a91aedbd1dfecc1b3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:28.683951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] RBAC: Authorizing `identity:delete_user(user_id=968296b8db55464a91aedbd1dfecc1b3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:28.684136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:28.893461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968296b8db55464a91aedbd1dfecc1b3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:28.894471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968296b8db55464a91aedbd1dfecc1b3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:28.904447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968296b8db55464a91aedbd1dfecc1b3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:28.988404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968296b8db55464a91aedbd1dfecc1b3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:28.990899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:29.002358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968296b8db55464a91aedbd1dfecc1b3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:29.008946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 983/1992] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:28 2022] DELETE /identity/v3/users/968296b8db55464a91aedbd1dfecc1b3 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:29.013548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:29.014032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:29.014122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:29.033716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:29.034933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:29.094313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1010/1993] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:29.104837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:29.106442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:29.168866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e680d49-6ae0-4338-8727-3f3fa41538da admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:29.172468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:29.173103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:29.173984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:29.174233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:29.183419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:29.184937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:29.197518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 984/1994] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:29.756454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:29.758103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:29.772686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afcea0-227c-4c96-9b21-7a421c68ffa0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:29.778238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:29.779141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:29.779465 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:29.779753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] PATH_INFO: `/v3/projects/d43f7e6aca9a47c38799a19001c387f4` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:29.781015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d43f7e6aca9a47c38799a19001c387f4)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:29.782952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:29.894726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd43f7e6aca9a47c38799a19001c387f4', 'request_id': 'req-07f70949-b7d7-4b66-96e1-cbe3284d4a17'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:29.960085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd43f7e6aca9a47c38799a19001c387f4', 'request_id': 'req-07f70949-b7d7-4b66-96e1-cbe3284d4a17'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:29.974523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd43f7e6aca9a47c38799a19001c387f4', 'request_id': 'req-07f70949-b7d7-4b66-96e1-cbe3284d4a17'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.000552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] The token cache is being invalidate because project d43f7e6aca9a47c38799a19001c387f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:30.000850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.002066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.003676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1011/1995] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:29 2022] DELETE /identity/v3/projects/d43f7e6aca9a47c38799a19001c387f4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:30.007885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:30.018740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8701615-8f72-4a99-bf1f-f8ab35fe7dc0 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:30.080229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:30.081575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:30.081860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:30.082142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] PATH_INFO: `/v3/users/f9ee227fa81d4a03b98b609f738c5c47` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:30.083583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ee227fa81d4a03b98b609f738c5c47)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:30.085554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:30.340977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ee227fa81d4a03b98b609f738c5c47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.342050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ee227fa81d4a03b98b609f738c5c47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.350632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ee227fa81d4a03b98b609f738c5c47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.428148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ee227fa81d4a03b98b609f738c5c47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.443099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7641de09-9dfb-4ec8-b99e-faecd77d5fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ee227fa81d4a03b98b609f738c5c47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:30.447575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 985/1996] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:30 2022] DELETE /identity/v3/users/f9ee227fa81d4a03b98b609f738c5c47 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:30.920867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:30.923215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:30.941248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f70949-b7d7-4b66-96e1-cbe3284d4a17 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:31.002350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:31.003014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.003333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.003617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] PATH_INFO: `/v3/projects/0e50fa0ee0fd4d8baa9813c113bd43dc` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.004843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e50fa0ee0fd4d8baa9813c113bd43dc)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:31.006819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:31.015881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:31.120916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e50fa0ee0fd4d8baa9813c113bd43dc', 'request_id': 'req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:31.217522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e50fa0ee0fd4d8baa9813c113bd43dc', 'request_id': 'req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:31.224071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e50fa0ee0fd4d8baa9813c113bd43dc', 'request_id': 'req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:31.236787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] The token cache is being invalidate because project 0e50fa0ee0fd4d8baa9813c113bd43dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:31.237217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:31.237889 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5b3ecdd8-e7cf-4ed3-97fd-0808293c0cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:31.239569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1012/1997] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:30 2022] DELETE /identity/v3/projects/0e50fa0ee0fd4d8baa9813c113bd43dc => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:31.245570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98fbd7-eb9d-4398-ae4f-9d2dad05e954 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.254691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98fbd7-eb9d-4398-ae4f-9d2dad05e954 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.254691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98fbd7-eb9d-4398-ae4f-9d2dad05e954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.256949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:31.269396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8e98fbd7-eb9d-4398-ae4f-9d2dad05e954 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:31.270635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8e98fbd7-eb9d-4398-ae4f-9d2dad05e954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:31.334435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 986/1998] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:31.343060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524963d-7eee-428d-89ae-968f3cccbfb6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.343366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524963d-7eee-428d-89ae-968f3cccbfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.343658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524963d-7eee-428d-89ae-968f3cccbfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.365866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d524963d-7eee-428d-89ae-968f3cccbfb6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:31.367105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d524963d-7eee-428d-89ae-968f3cccbfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:31.375283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1013/1999] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:31.380578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98fbd7-eb9d-4398-ae4f-9d2dad05e954 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:31.390680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98fbd7-eb9d-4398-ae4f-9d2dad05e954 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:31.396135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:31.396788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.397074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.397355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.398157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:31.398467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:31.400130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:31.412313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 987/2000] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:31.417113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524963d-7eee-428d-89ae-968f3cccbfb6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:31.426896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524963d-7eee-428d-89ae-968f3cccbfb6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:31.432200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:31.437887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.437887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.437887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.437887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:31.437887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:31.536929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1014/2001] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:31 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:31.538336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:31.549053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcce01-ee94-47e6-bdef-e2a9d4e8fdfb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:31.608182 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:31.608810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.609191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.609398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.610090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:31.612907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:31.749135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 988/2002] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:31 2022] POST /identity/v3/users => generated 279 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:31.754119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:31.765160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5830c9-58dd-4ce3-86bd-baa65c2b3f5c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:31.769957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:31.770608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.770900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.771215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.771956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:31.773610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:31.784870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1015/2003] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:31.789580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:31.800614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3759cd-849c-4028-8ed7-9454a3e91230 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:31.805476 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b84af7-a4a3-476d-bab6-754a7680f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:31.806124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84af7-a4a3-476d-bab6-754a7680f7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.806413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84af7-a4a3-476d-bab6-754a7680f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.806690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84af7-a4a3-476d-bab6-754a7680f7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.807810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b84af7-a4a3-476d-bab6-754a7680f7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:31.809473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b84af7-a4a3-476d-bab6-754a7680f7e9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:31.822371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 989/2004] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:31.827023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:31.837359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb9108-35d0-4b12-81d9-0513a42d27ff admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:31.843302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:31.843980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:31.844266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:31.844551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] PATH_INFO: `/v3/projects/82920a312dfb4338872c376816c742a3/users/0446afe44d5c4a518d35c5520d3e869a/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:31.860276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82920a312dfb4338872c376816c742a3, user_id=0446afe44d5c4a518d35c5520d3e869a, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:31.863724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:32.048011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1016/2005] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:31 2022] PUT /identity/v3/projects/82920a312dfb4338872c376816c742a3/users/0446afe44d5c4a518d35c5520d3e869a/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:32.069681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b2ea-d0ee-430a-a85f-e0e1d5b01903 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.070123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b2ea-d0ee-430a-a85f-e0e1d5b01903 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.070855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b2ea-d0ee-430a-a85f-e0e1d5b01903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.100817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5e04b2ea-d0ee-430a-a85f-e0e1d5b01903 None None] MFA Rules not processed for user `0446afe44d5c4a518d35c5520d3e869a`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:32.102027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5e04b2ea-d0ee-430a-a85f-e0e1d5b01903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:32.164709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:32.168353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 990/2006] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:32.172989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:32.242043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec1b2c-be76-4da6-8e8e-7b4d1e3b69c9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:32.247176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:32.247867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.248155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.253211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.253211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:32.253211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:32.350277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1017/2007] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:32 2022] POST /identity/v3/projects => generated 357 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:32.360616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b2ea-d0ee-430a-a85f-e0e1d5b01903 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:32.373821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b2ea-d0ee-430a-a85f-e0e1d5b01903 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:32.437517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:32.438152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.438484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.438725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.439615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:32.445610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:32.557998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 991/2008] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:32 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:32.559394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:32.570173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306af7aa-39fc-419b-a93a-102c55f96343 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:32.579997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:32.580654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.580990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.581299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.582153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:32.583635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:32.595257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1018/2009] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:32.600372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:32.610791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09458b36-b07a-4eed-a82e-0eefe1cc1f69 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:32.615895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:32.616584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.616903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.617327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] PATH_INFO: `/v3/projects/967d7ca5b5f8499da4229d6e489fca4a/users/c29e2d70f5604d1dbcea3839540c9555/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.633711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967d7ca5b5f8499da4229d6e489fca4a, user_id=c29e2d70f5604d1dbcea3839540c9555, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:32.636482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:32.772858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 992/2010] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:32 2022] PUT /identity/v3/projects/967d7ca5b5f8499da4229d6e489fca4a/users/c29e2d70f5604d1dbcea3839540c9555/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:32.775502 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:32.786480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f5919-e168-4b5b-b72c-312d96bc7215 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:32.790547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:32.848479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904f171-0c31-4e53-b181-2606b3f9b8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:32.849120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904f171-0c31-4e53-b181-2606b3f9b8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.849464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904f171-0c31-4e53-b181-2606b3f9b8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.849739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904f171-0c31-4e53-b181-2606b3f9b8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.850474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904f171-0c31-4e53-b181-2606b3f9b8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:32.852095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904f171-0c31-4e53-b181-2606b3f9b8b7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:32.862535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1019/2011] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:32.867464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:32.878780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58976554-97b9-4081-9a03-14bb83aa5633 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:32.883999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:32.884671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.884959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.885240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] PATH_INFO: `/v3/projects/967d7ca5b5f8499da4229d6e489fca4a/users/c29e2d70f5604d1dbcea3839540c9555/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.887273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967d7ca5b5f8499da4229d6e489fca4a, user_id=c29e2d70f5604d1dbcea3839540c9555, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:32.890093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:32.901659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 993/2012] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:32 2022] PUT /identity/v3/projects/967d7ca5b5f8499da4229d6e489fca4a/users/c29e2d70f5604d1dbcea3839540c9555/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:32.912810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:32.913105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:32.913387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:32.944645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] MFA Rules not processed for user `c29e2d70f5604d1dbcea3839540c9555`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:32.945924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:33.015503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1020/2013] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:33.027406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:33.029190 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:33.094430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86bbe7-17f1-4ba2-bd3e-337ebf455676 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:33.098216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:33.098870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:33.099155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:33.099469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:33.109625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:33.111915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:33.123977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 994/2014] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:33.377900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:33.379843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:33.396642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188d25e-5dbf-48c8-80e4-578a844722e1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:33.400960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0d1a9-3e35-49fb-bfbe-6f7d1fbe6a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:33.401688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d1a9-3e35-49fb-bfbe-6f7d1fbe6a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:33.402188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d1a9-3e35-49fb-bfbe-6f7d1fbe6a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:33.402536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d1a9-3e35-49fb-bfbe-6f7d1fbe6a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:33.476270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0d1a9-3e35-49fb-bfbe-6f7d1fbe6a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:33.477834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0d1a9-3e35-49fb-bfbe-6f7d1fbe6a5a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:33.488041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1021/2015] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:33.796905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:33.799019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:33.888395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:33.896144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a6b59-bbd1-43d8-bc8a-01b083a0546a service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:33.901862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:33.902598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:33.902898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:33.905139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] PATH_INFO: `/v3/users/063fbcde0b8b4d49b103b307723e49af` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:33.905287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=063fbcde0b8b4d49b103b307723e49af)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:33.907283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:34.185452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063fbcde0b8b4d49b103b307723e49af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:34.186311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063fbcde0b8b4d49b103b307723e49af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:34.201222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063fbcde0b8b4d49b103b307723e49af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:34.315067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063fbcde0b8b4d49b103b307723e49af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:34.331086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063fbcde0b8b4d49b103b307723e49af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:34.337467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 995/2016] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:33 2022] DELETE /identity/v3/users/063fbcde0b8b4d49b103b307723e49af => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:34.342042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:34.342333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:34.342612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:34.368486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:34.369859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:34.395379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:34.459160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1022/2017] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:34.476710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:34.478327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:34.558121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7655b9-4656-44a7-8344-a834bd739248 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:34.561878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:34.562492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:34.562780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:34.563064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:34.572961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:34.574432 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:34.584978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 996/2018] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:35.076355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:35.078100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:35.099838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aaaba5-5224-4970-af95-85adfa13621d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:35.108837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:35.109872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:35.109872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:35.110175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] PATH_INFO: `/v3/projects/560780ba854d4df9a8ee83369de74a2e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:35.112868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] RBAC: Authorizing `identity:delete_project(project_id=560780ba854d4df9a8ee83369de74a2e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:35.115999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:35.318384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560780ba854d4df9a8ee83369de74a2e', 'request_id': 'req-52774e5d-e49c-450e-a590-06192002414d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:35.503039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560780ba854d4df9a8ee83369de74a2e', 'request_id': 'req-52774e5d-e49c-450e-a590-06192002414d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:35.533885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560780ba854d4df9a8ee83369de74a2e', 'request_id': 'req-52774e5d-e49c-450e-a590-06192002414d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:35.568501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] The token cache is being invalidate because project 560780ba854d4df9a8ee83369de74a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:35.568849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:35.571673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:35.584804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1023/2019] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:35 2022] DELETE /identity/v3/projects/560780ba854d4df9a8ee83369de74a2e => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:35.588767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:35.599685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fed7c5-0603-4181-bc6c-3bae8ff6410d service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:35.693007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:35.693674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:35.693962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:35.694244 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] PATH_INFO: `/v3/users/108eae1a28bd4a18a95ad25965a32c3c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:35.695473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108eae1a28bd4a18a95ad25965a32c3c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:35.697509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:35.719661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:35.868817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:35.870241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:35.883603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52774e5d-e49c-450e-a590-06192002414d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:35.889008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:35.889672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:35.890065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:35.890239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] PATH_INFO: `/v3/users/19164ea012924460a1508132dd8aee49` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:35.891437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=19164ea012924460a1508132dd8aee49)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:35.893356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:35.978851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108eae1a28bd4a18a95ad25965a32c3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:35.979947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108eae1a28bd4a18a95ad25965a32c3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:35.992907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108eae1a28bd4a18a95ad25965a32c3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.108515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108eae1a28bd4a18a95ad25965a32c3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.122624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108eae1a28bd4a18a95ad25965a32c3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.124075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19164ea012924460a1508132dd8aee49'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.129022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 997/2020] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:35 2022] DELETE /identity/v3/users/108eae1a28bd4a18a95ad25965a32c3c => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:36.131301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19164ea012924460a1508132dd8aee49'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.132137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19164ea012924460a1508132dd8aee49'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.132761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19164ea012924460a1508132dd8aee49'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.134746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:36.135895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:36.147621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6495506b-c0ec-45a6-97ac-07ed76abb1eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19164ea012924460a1508132dd8aee49'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.207524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54158e65-ffd2-4039-adf0-501a10c1c973 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:36.214217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:36.215038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:36.215417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:36.215825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] PATH_INFO: `/v3/users/00968a5e985e4b29861816170b870464` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:36.217360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=00968a5e985e4b29861816170b870464)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:36.220935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:36.249066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1024/2021] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:35 2022] DELETE /identity/v3/users/19164ea012924460a1508132dd8aee49 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:36.253900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:36.254194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:36.254481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:36.273961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:36.275203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:36.292858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:36.339082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1025/2022] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:36.349259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:36.350614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:36.414954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9350bc3-cac1-4be2-9dd5-c660b0af11b6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:36.418882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af5612-45d1-46ff-9ab4-549f506ccdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:36.425010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af5612-45d1-46ff-9ab4-549f506ccdfe service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:36.425010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af5612-45d1-46ff-9ab4-549f506ccdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:36.425010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af5612-45d1-46ff-9ab4-549f506ccdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:36.430553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af5612-45d1-46ff-9ab4-549f506ccdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:36.432171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af5612-45d1-46ff-9ab4-549f506ccdfe service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:36.444665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1026/2023] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:36.463027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00968a5e985e4b29861816170b870464'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.463869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00968a5e985e4b29861816170b870464'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.472216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00968a5e985e4b29861816170b870464'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.544778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00968a5e985e4b29861816170b870464'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.563712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00968a5e985e4b29861816170b870464'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.571485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 998/2024] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:36 2022] DELETE /identity/v3/users/00968a5e985e4b29861816170b870464 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:36.576255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:36.576715 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:36.576888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:36.608296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:36.609776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:36.653801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:36.655825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:36.694372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:36.734925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1027/2025] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:36.748373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:36.750300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:36.798946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08211b-0194-4136-9676-08b15e90eeeb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:36.812803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:36.814583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:36.814877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:36.815492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] PATH_INFO: `/v3/projects/bccf7f2ae40a4233916dd5ff39304736` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:36.816940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccf7f2ae40a4233916dd5ff39304736)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:36.820214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:36.854553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c005a7-d264-4b0e-9135-b68530c82df4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:36.861491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:36.863426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:36.863426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:36.863626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:36.877876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:36.879394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:36.890035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1028/2026] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:36.952709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccf7f2ae40a4233916dd5ff39304736', 'request_id': 'req-b8ed9a16-6448-4c85-abd6-aca3dbd29790'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.960091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccf7f2ae40a4233916dd5ff39304736', 'request_id': 'req-b8ed9a16-6448-4c85-abd6-aca3dbd29790'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:36.979244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:36.980804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:36.997832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6caea4-96b2-4685-bc24-b6bdf926874e service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.004195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:37.059975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.060690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.062163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccf7f2ae40a4233916dd5ff39304736', 'request_id': 'req-b8ed9a16-6448-4c85-abd6-aca3dbd29790'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.063690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.064016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] PATH_INFO: `/v3/projects/2562f743952349e4a4bb310e0ede0e0a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.065286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2562f743952349e4a4bb310e0ede0e0a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.067269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.075666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] The token cache is being invalidate because project bccf7f2ae40a4233916dd5ff39304736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:37.076012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.076654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8ed9a16-6448-4c85-abd6-aca3dbd29790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.078252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 999/2027] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:36 2022] DELETE /identity/v3/projects/bccf7f2ae40a4233916dd5ff39304736 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:37.083682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2da3c-3b16-4d48-aabf-c7fb3d245ef7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.084226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2da3c-3b16-4d48-aabf-c7fb3d245ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.084632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2da3c-3b16-4d48-aabf-c7fb3d245ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.105739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-33b2da3c-3b16-4d48-aabf-c7fb3d245ef7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:37.107004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-33b2da3c-3b16-4d48-aabf-c7fb3d245ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.115741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1000/2028] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:37.124090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e9f49-87b1-4d8c-b172-5dd71be2cf78 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.124683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e9f49-87b1-4d8c-b172-5dd71be2cf78 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.125164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e9f49-87b1-4d8c-b172-5dd71be2cf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.150611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f23e9f49-87b1-4d8c-b172-5dd71be2cf78 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:37.151976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f23e9f49-87b1-4d8c-b172-5dd71be2cf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.161643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1001/2029] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:37.165705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e9f49-87b1-4d8c-b172-5dd71be2cf78 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.177163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e9f49-87b1-4d8c-b172-5dd71be2cf78 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.188105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.188775 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.189558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.189968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.190617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:37.191068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.192848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.211845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1002/2030] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:22:37.223001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.241317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd84660-f481-4af3-b33b-c0762be3608d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.249332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.250507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.251093 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.252175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.253422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.257239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.274204 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2562f743952349e4a4bb310e0ede0e0a', 'request_id': 'req-46d3338b-eb74-40c9-9fef-4584185a9800'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.388278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2562f743952349e4a4bb310e0ede0e0a', 'request_id': 'req-46d3338b-eb74-40c9-9fef-4584185a9800'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.398467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2562f743952349e4a4bb310e0ede0e0a', 'request_id': 'req-46d3338b-eb74-40c9-9fef-4584185a9800'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.401410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1003/2031] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:37 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:37.403106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.411150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] The token cache is being invalidate because project 2562f743952349e4a4bb310e0ede0e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:37.411639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.412509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-46d3338b-eb74-40c9-9fef-4584185a9800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.414395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1029/2032] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:36 2022] DELETE /identity/v3/projects/2562f743952349e4a4bb310e0ede0e0a => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:37.419855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.420361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.420818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.443512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:37.445462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.485269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef78833-5aa7-410b-8049-b5b20d775417 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.492239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbe20d-2aa8-4142-9ea3-a31d2ca82bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.492889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbe20d-2aa8-4142-9ea3-a31d2ca82bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.493180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbe20d-2aa8-4142-9ea3-a31d2ca82bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.493464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbe20d-2aa8-4142-9ea3-a31d2ca82bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.494700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbe20d-2aa8-4142-9ea3-a31d2ca82bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.496739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbe20d-2aa8-4142-9ea3-a31d2ca82bb4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.526065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1030/2033] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:37.533589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.533589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.545509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f759e-59ab-4a9f-9816-66ab9e7c7ca9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.550806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.551512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.552083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.552192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] PATH_INFO: `/v3/projects/e893d04bd50d40bd9c69f1b624dfe2c9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.553371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e893d04bd50d40bd9c69f1b624dfe2c9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.556534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.691111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1004/2034] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:37 2022] POST /identity/v3/users => generated 295 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:37.691111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.691111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.691111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.704189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:37.711745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:37.713320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.719884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e893d04bd50d40bd9c69f1b624dfe2c9', 'request_id': 'req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.779025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1005/2035] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:37.780160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.781445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.793243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9019c3d-18f9-4797-b5c5-fa9244bbaaa0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.797959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.799104 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.799419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.799704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.800404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.802027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.812376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1006/2036] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:37.813348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.817156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e893d04bd50d40bd9c69f1b624dfe2c9', 'request_id': 'req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.823996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e893d04bd50d40bd9c69f1b624dfe2c9', 'request_id': 'req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.824276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107ba9-751e-40ef-aabd-6b90add2f660 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.829572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.830196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.830483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.830797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.831580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:37.831976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.833575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.836318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] The token cache is being invalidate because project e893d04bd50d40bd9c69f1b624dfe2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:37.836643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.837321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:37.839001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1031/2037] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:37 2022] DELETE /identity/v3/projects/e893d04bd50d40bd9c69f1b624dfe2c9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:37.839855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.841342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.849067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1007/2038] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:37.849994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.851822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.855917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0044d5-a5c2-4dec-883b-1c8c63e152b3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.862951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.863629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.863915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.864194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.864900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.866589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.872252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d375f6-3c26-477d-979f-3f119cc14c2e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.878169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1032/2039] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:37.879180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:37.879630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.881052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.881052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.881269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:37.881361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] PATH_INFO: `/v3/users/5d2e5ab58d634a6f90d3761cbb32ca11` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.882861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2e5ab58d634a6f90d3761cbb32ca11)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.885054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:37.894165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415309fe-e239-4ce3-98b8-0a0af37ab14b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:37.910286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:37.911301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:37.911724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:37.912203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:37.914700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:37.917446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.012888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1033/2040] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:37 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:38.013790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.081922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1674983-3c02-45a3-aa1e-a1ea59ffd5af admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.086967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.087773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.088061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.088343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] PATH_INFO: `/v3/projects/3c149f9b8d5d4c93bcf6ce1421ee9ea0/users/952968c8e1504229b93f316b1f8c6c2c/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.104921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c149f9b8d5d4c93bcf6ce1421ee9ea0, user_id=952968c8e1504229b93f316b1f8c6c2c, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.107162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2e5ab58d634a6f90d3761cbb32ca11'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:38.107834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.108188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2e5ab58d634a6f90d3761cbb32ca11'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:38.116788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2e5ab58d634a6f90d3761cbb32ca11'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:38.189262 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:38.217173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2e5ab58d634a6f90d3761cbb32ca11'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:38.224634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1034/2041] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:38 2022] PUT /identity/v3/projects/3c149f9b8d5d4c93bcf6ce1421ee9ea0/users/952968c8e1504229b93f316b1f8c6c2c/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:38.225522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.227160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:38.232295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4c4cac24-a1a7-4acd-8b1e-37034e4e9a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2e5ab58d634a6f90d3761cbb32ca11'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:38.244071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1008/2042] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:37 2022] DELETE /identity/v3/users/5d2e5ab58d634a6f90d3761cbb32ca11 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:38.245485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.245808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.246044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.303750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] MFA Rules not processed for user `952968c8e1504229b93f316b1f8c6c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:38.305466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:38.338519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8a1ee-1a6b-4d01-b097-440b58509b04 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.400951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1009/2043] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:38.406832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.407287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:38.428249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.431667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.431967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.432253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.438131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.438131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.492802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecd3c5-99cc-4b19-88e4-db957ad9d0ce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.497609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.498234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.498517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.498792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.499580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.501611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.606079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1035/2044] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:38 2022] POST /identity/v3/users => generated 291 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:38.608282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.624793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39848065-5c80-4c51-9b58-6296749daf30 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.631073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.631882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.632366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.632836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.633867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.635892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.640294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1010/2045] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:38 2022] POST /identity/v3/projects => generated 389 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:38.646763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.654016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1036/2046] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:38.659101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.671616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb693180-8b22-4233-a9a5-3749aa37b4d7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.724982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960701-6cc4-4522-8c2d-56d8bff5a077 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.732028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.733246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.733475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.733756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.734507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.737010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.757601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.758489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.759032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.759401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.760171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.762797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.779331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1037/2047] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:38.780513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.792238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40db80-8d4b-49c0-b889-f26001a30686 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.798288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf58e4-aa11-4f82-841f-2e1817cb2c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.799059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf58e4-aa11-4f82-841f-2e1817cb2c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.799595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf58e4-aa11-4f82-841f-2e1817cb2c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.800050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf58e4-aa11-4f82-841f-2e1817cb2c62 admin admin] PATH_INFO: `/v3/projects/356ad325d7914771a7e1b66caf2b6e4d/users/1a800d95ac1d492bb44947fdf5bb3377/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.835581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1011/2048] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:38 2022] POST /identity/v3/users => generated 301 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:38.838682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.840458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf58e4-aa11-4f82-841f-2e1817cb2c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356ad325d7914771a7e1b66caf2b6e4d, user_id=1a800d95ac1d492bb44947fdf5bb3377, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.843786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf58e4-aa11-4f82-841f-2e1817cb2c62 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.857648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923131fd-04f1-465a-9376-78989d83140f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.865616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.866574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.867420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.867893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.868928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.871056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.887827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1012/2049] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:38.893541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:38.905829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56262f72-44e4-4461-b95a-d199c59f50bb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:38.919009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:38.919009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.919009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.919009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] PATH_INFO: `/v3/projects/4fac8a5e62f74bc9b91b6e5b787a4df4/users/15e35aebe7e64639aee33c34ad1f6b16/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.947341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1038/2050] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:38 2022] PUT /identity/v3/projects/356ad325d7914771a7e1b66caf2b6e4d/users/1a800d95ac1d492bb44947fdf5bb3377/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:38.952405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:38.952711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:38.952998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:38.959601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fac8a5e62f74bc9b91b6e5b787a4df4, user_id=15e35aebe7e64639aee33c34ad1f6b16, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:38.962731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:38.979747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:38.981067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:39.054258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1039/2051] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:39.065557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:39.067240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:39.086660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1013/2052] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:38 2022] PUT /identity/v3/projects/4fac8a5e62f74bc9b91b6e5b787a4df4/users/15e35aebe7e64639aee33c34ad1f6b16/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:39.093946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:39.154150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caff51-f7a9-4906-8b55-dc5e9b72b360 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:39.203784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:39.238315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:39.239094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.239356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f1b89-f3d3-4c59-ad29-98ddfed15ab6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:39.239545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.243749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.250999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:39.253253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.253917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.255106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.256556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:39.258322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:39.258829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:39.259910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:39.277935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1014/2053] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:39.279040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:39.282872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:39.286861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1040/2054] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:39.288244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:39.307434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237e4c4-d482-4fe6-8d53-5b48d8dd2fcc service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:39.307434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea35d1-baa0-4a21-a19a-42a40c92aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:39.307434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea35d1-baa0-4a21-a19a-42a40c92aa41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.307434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea35d1-baa0-4a21-a19a-42a40c92aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.308659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea35d1-baa0-4a21-a19a-42a40c92aa41 admin admin] PATH_INFO: `/v3/projects/4fac8a5e62f74bc9b91b6e5b787a4df4/users/15e35aebe7e64639aee33c34ad1f6b16/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.309539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea35d1-baa0-4a21-a19a-42a40c92aa41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fac8a5e62f74bc9b91b6e5b787a4df4, user_id=15e35aebe7e64639aee33c34ad1f6b16, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:39.312231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e879d8-d723-42b2-bf64-8918536fb861 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:39.312985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea35d1-baa0-4a21-a19a-42a40c92aa41 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:39.327918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1041/2055] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:39 2022] PUT /identity/v3/projects/4fac8a5e62f74bc9b91b6e5b787a4df4/users/15e35aebe7e64639aee33c34ad1f6b16/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:39.330425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:39.331980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.332957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.333464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] PATH_INFO: `/v3/projects/e8c6508f19414c39a4b22525dc3a0360` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.335124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c6508f19414c39a4b22525dc3a0360)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:39.342062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:39.353511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.354334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.354834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.398881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] MFA Rules not processed for user `15e35aebe7e64639aee33c34ad1f6b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:39.400190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:39.479484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1042/2056] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:39 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:39.492201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:39.493669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:39.562292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:39.581703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03932ca-dd87-4e9b-b606-c8232ff7efd7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:39.590857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c6508f19414c39a4b22525dc3a0360', 'request_id': 'req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:39.608193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c6508f19414c39a4b22525dc3a0360', 'request_id': 'req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:39.655850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218f353-b13e-4e01-9f6e-ffdf48a51f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:39.656659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218f353-b13e-4e01-9f6e-ffdf48a51f8f service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.658089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218f353-b13e-4e01-9f6e-ffdf48a51f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.658421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218f353-b13e-4e01-9f6e-ffdf48a51f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.719763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c6508f19414c39a4b22525dc3a0360', 'request_id': 'req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:39.734288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] The token cache is being invalidate because project e8c6508f19414c39a4b22525dc3a0360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:39.735390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:39.736565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-46a0c22b-13ec-4d8e-8c58-ae3424f22f3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:39.744779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1015/2057] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:39 2022] DELETE /identity/v3/projects/e8c6508f19414c39a4b22525dc3a0360 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:39.746318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f81ef-31a5-47f8-bfeb-a1122814b8dc None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.746763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f81ef-31a5-47f8-bfeb-a1122814b8dc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.747304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f81ef-31a5-47f8-bfeb-a1122814b8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.772744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218f353-b13e-4e01-9f6e-ffdf48a51f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:39.774935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218f353-b13e-4e01-9f6e-ffdf48a51f8f service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:39.780252 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-458f81ef-31a5-47f8-bfeb-a1122814b8dc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:39.781909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-458f81ef-31a5-47f8-bfeb-a1122814b8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:39.792180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1043/2058] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:39 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:39.891487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1016/2059] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:39.894411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e379a18-c6d1-4ba0-9c00-1fdba7d9851a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.894411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e379a18-c6d1-4ba0-9c00-1fdba7d9851a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.894411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e379a18-c6d1-4ba0-9c00-1fdba7d9851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.923727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3e379a18-c6d1-4ba0-9c00-1fdba7d9851a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:39.925800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3e379a18-c6d1-4ba0-9c00-1fdba7d9851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:39.935178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1044/2060] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:39.939936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f81ef-31a5-47f8-bfeb-a1122814b8dc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:39.953216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f81ef-31a5-47f8-bfeb-a1122814b8dc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:39.966457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:39.967140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:39.967527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:39.967833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:39.968793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:39.969116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:39.976551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:39.996517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1017/2061] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:22:40.000896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e379a18-c6d1-4ba0-9c00-1fdba7d9851a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.011301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e379a18-c6d1-4ba0-9c00-1fdba7d9851a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.017954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.018527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.018886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.019267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.020041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.022159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.028360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.031031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:40.053394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf7484-5de8-423a-a1ad-6624b6276f40 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.058861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.059853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.060360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.060737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] PATH_INFO: `/v3/users/952968c8e1504229b93f316b1f8c6c2c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.062175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952968c8e1504229b93f316b1f8c6c2c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.064331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.130224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1045/2062] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:39 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:40.136181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.260317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ae5db-b283-485d-b79b-9a699550861c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.264558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952968c8e1504229b93f316b1f8c6c2c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:40.265269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952968c8e1504229b93f316b1f8c6c2c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:40.270608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.271480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.271777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.272116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.273510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.276517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.282034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952968c8e1504229b93f316b1f8c6c2c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:40.368296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952968c8e1504229b93f316b1f8c6c2c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:40.389393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3738ec3d-1b4a-4421-a7bb-77dd6981fa16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952968c8e1504229b93f316b1f8c6c2c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:40.400875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1018/2063] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:40 2022] DELETE /identity/v3/users/952968c8e1504229b93f316b1f8c6c2c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:40.402931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.403356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.403967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.410709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1046/2064] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:40 2022] POST /identity/v3/users => generated 291 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:40.414986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.419612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:40.429591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:40.430871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:40.443922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be374c86-9a3a-4210-ba9c-469d8a4384c3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.525327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1019/2065] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:40.537347 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.537961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.538251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.538548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.539486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.541241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.545500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.547242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:40.560412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1047/2066] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:40.561528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.573742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3960b-e60a-4e92-9393-f0a60f07d77a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.578842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.580395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.580724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.581164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.582098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.584657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.611708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1048/2067] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:40.614026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:40.628747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c64d34-31d8-4c42-9f3d-4d4e6e294826 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.633102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccb6b6-615e-4919-a3f5-5e783bd0964c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:40.633967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4755a-26a9-451c-8e2e-71f71b90b08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.635241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4755a-26a9-451c-8e2e-71f71b90b08a service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.635701 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4755a-26a9-451c-8e2e-71f71b90b08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.636173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4755a-26a9-451c-8e2e-71f71b90b08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.639054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:40.639985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.640432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.640777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] PATH_INFO: `/v3/projects/45e265d81c4d4cac9cb32ed1d95516f8/users/ec3272220b564990b9f62896bcd4e158/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.648847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4755a-26a9-451c-8e2e-71f71b90b08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.650394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4755a-26a9-451c-8e2e-71f71b90b08a service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.662149 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1020/2068] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:40.663502 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e265d81c4d4cac9cb32ed1d95516f8, user_id=ec3272220b564990b9f62896bcd4e158, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:40.666448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:40.749944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1049/2069] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:40 2022] PUT /identity/v3/projects/45e265d81c4d4cac9cb32ed1d95516f8/users/ec3272220b564990b9f62896bcd4e158/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:40.781139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:40.781647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:40.781723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:40.899424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:40.904780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] MFA Rules not processed for user `ec3272220b564990b9f62896bcd4e158`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:40.910350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.034965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1021/2070] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:41.039321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.131721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a5c83-2778-44d3-bab9-dd7c93c9e698 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.136893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.137344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.137667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.137974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.138702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.141111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.274864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:41.276160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.278962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.295640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1050/2071] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:41 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:41.301121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.312260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718befdd-5f21-4354-aad6-d5959912ba89 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.433047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824727c-26de-488b-bf34-fe016188e55b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.433793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.435161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.435487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.435842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] PATH_INFO: `/v3/projects/3c149f9b8d5d4c93bcf6ce1421ee9ea0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.437186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c149f9b8d5d4c93bcf6ce1421ee9ea0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.439725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.449554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed8e8-8b25-4a87-85e8-8f8f9cb45c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.452759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed8e8-8b25-4a87-85e8-8f8f9cb45c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.452759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed8e8-8b25-4a87-85e8-8f8f9cb45c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.452759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed8e8-8b25-4a87-85e8-8f8f9cb45c73 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.452759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed8e8-8b25-4a87-85e8-8f8f9cb45c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.456137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed8e8-8b25-4a87-85e8-8f8f9cb45c73 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.608084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c149f9b8d5d4c93bcf6ce1421ee9ea0', 'request_id': 'req-4e91290b-8983-49d5-9226-619bfcc8d7a4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:41.615589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c149f9b8d5d4c93bcf6ce1421ee9ea0', 'request_id': 'req-4e91290b-8983-49d5-9226-619bfcc8d7a4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:41.643762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1051/2072] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:41 2022] POST /identity/v3/users => generated 296 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:41.643762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.643762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.643762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.673436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] MFA Rules not processed for user `1a800d95ac1d492bb44947fdf5bb3377`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:41.674691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.696751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c149f9b8d5d4c93bcf6ce1421ee9ea0', 'request_id': 'req-4e91290b-8983-49d5-9226-619bfcc8d7a4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:41.713129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] The token cache is being invalidate because project 3c149f9b8d5d4c93bcf6ce1421ee9ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:41.714264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:41.715035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:41.716952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1022/2073] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:41 2022] DELETE /identity/v3/projects/3c149f9b8d5d4c93bcf6ce1421ee9ea0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:41.718098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.721729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.757547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1052/2074] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:41.757547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.757547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.806634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91290b-8983-49d5-9226-619bfcc8d7a4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.813272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.815223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.815223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.815607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.816778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.818310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.831079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1023/2075] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:41.835884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.838534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:41.850102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd45aae-de6f-4b47-bdf9-55aa7a3da03f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.856465 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.857226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.857759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.858191 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] PATH_INFO: `/v3/users/15e35aebe7e64639aee33c34ad1f6b16` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.859577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e35aebe7e64639aee33c34ad1f6b16)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.861981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.874941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:41.905859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715021c-6ffa-48fc-9804-8c6aaf92ac9f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.914345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.915006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.915948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.915948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.925978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.927271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:41.939753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1024/2076] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:41.940603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:41.960443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92b53b-d16f-4e2b-8cf6-082e573c0171 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:41.967910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:41.968714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:41.969054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:41.969405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] PATH_INFO: `/v3/projects/0e2008842f754b13bec9a4d029f588f4/users/52818dc9985d4d7e94ea98674f285aa4/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:41.991554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2008842f754b13bec9a4d029f588f4, user_id=52818dc9985d4d7e94ea98674f285aa4, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:41.995519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:42.047695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e35aebe7e64639aee33c34ad1f6b16'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:42.057221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e35aebe7e64639aee33c34ad1f6b16'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:42.057994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e35aebe7e64639aee33c34ad1f6b16'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:42.058703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e35aebe7e64639aee33c34ad1f6b16'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:42.073920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e35aebe7e64639aee33c34ad1f6b16'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:42.083288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1025/2077] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:41 2022] PUT /identity/v3/projects/0e2008842f754b13bec9a4d029f588f4/users/52818dc9985d4d7e94ea98674f285aa4/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:42.087309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:42.089305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:42.119622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:42.178087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35006a5-847f-43e5-94de-e3113616bedc admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:42.181049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1053/2078] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:41 2022] DELETE /identity/v3/users/15e35aebe7e64639aee33c34ad1f6b16 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:42.254259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31380fc-1bdd-4e6e-85dc-979122e84094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:42.255250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31380fc-1bdd-4e6e-85dc-979122e84094 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:42.255509 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31380fc-1bdd-4e6e-85dc-979122e84094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:42.255793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31380fc-1bdd-4e6e-85dc-979122e84094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:42.256763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31380fc-1bdd-4e6e-85dc-979122e84094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:42.258502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31380fc-1bdd-4e6e-85dc-979122e84094 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:42.270321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1026/2079] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:42.274701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:42.286709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8340b-b528-449d-afe6-73936b598d74 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:42.291601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:42.292299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:42.292601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:42.292863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] PATH_INFO: `/v3/projects/0e2008842f754b13bec9a4d029f588f4/users/52818dc9985d4d7e94ea98674f285aa4/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:42.294876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2008842f754b13bec9a4d029f588f4, user_id=52818dc9985d4d7e94ea98674f285aa4, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:42.297895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:42.310105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1054/2080] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:42 2022] PUT /identity/v3/projects/0e2008842f754b13bec9a4d029f588f4/users/52818dc9985d4d7e94ea98674f285aa4/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:42.321902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:42.322199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:42.322483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:42.357052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] MFA Rules not processed for user `52818dc9985d4d7e94ea98674f285aa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:42.358382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:42.473599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1027/2081] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:42.492671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:42.494887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:42.590623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f4d01-228c-4f9f-95ec-5355bc360ba5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:42.595729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:42.596390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:42.596673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:42.596945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:42.607907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:42.609861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:42.624694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1055/2082] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:42.867981 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:42.877702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:42.985075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:43.041775 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb60c84-d7ae-4d43-a494-9204183a5346 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:43.055631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:43.056316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.056603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.059349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] PATH_INFO: `/v3/projects/4fac8a5e62f74bc9b91b6e5b787a4df4` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.059349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fac8a5e62f74bc9b91b6e5b787a4df4)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:43.063280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:43.207720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fac8a5e62f74bc9b91b6e5b787a4df4', 'request_id': 'req-62744df4-e074-46e3-a6b4-55f632f0fb97'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.219256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fac8a5e62f74bc9b91b6e5b787a4df4', 'request_id': 'req-62744df4-e074-46e3-a6b4-55f632f0fb97'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.306584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fac8a5e62f74bc9b91b6e5b787a4df4', 'request_id': 'req-62744df4-e074-46e3-a6b4-55f632f0fb97'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.324409 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] The token cache is being invalidate because project 4fac8a5e62f74bc9b91b6e5b787a4df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:43.325539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.327653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62744df4-e074-46e3-a6b4-55f632f0fb97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.335284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:43.337540 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1028/2083] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:42 2022] DELETE /identity/v3/projects/4fac8a5e62f74bc9b91b6e5b787a4df4 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:43.346586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e56e5-c65c-480b-930e-022ce51d50d7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.348168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e56e5-c65c-480b-930e-022ce51d50d7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.348622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e56e5-c65c-480b-930e-022ce51d50d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.352104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a776ac-bba9-4b75-ae7e-0ace693da4ce service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:43.382943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c38e56e5-c65c-480b-930e-022ce51d50d7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:43.384222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c38e56e5-c65c-480b-930e-022ce51d50d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:43.453187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:43.535214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:43.536345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.536731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.537117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] PATH_INFO: `/v3/users/ec3272220b564990b9f62896bcd4e158` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.539523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec3272220b564990b9f62896bcd4e158)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:43.544412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:43.581616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1029/2084] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:43.589618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b508-d6e1-4b81-814c-bc50ced29b5b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.589992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b508-d6e1-4b81-814c-bc50ced29b5b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.590508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b508-d6e1-4b81-814c-bc50ced29b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.614197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-40f0b508-d6e1-4b81-814c-bc50ced29b5b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:43.615645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-40f0b508-d6e1-4b81-814c-bc50ced29b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:43.628954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1030/2085] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:43.630078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0b508-d6e1-4b81-814c-bc50ced29b5b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:43.646474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0b508-d6e1-4b81-814c-bc50ced29b5b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:43.655337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:43.656635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.657580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.657727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.659225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:43.659905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:43.662521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:43.682850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1031/2086] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:43.685596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:43.703710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3f27e-0ed7-433f-b15d-8beb0888e31a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:43.711122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:43.717674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.717674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.717674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.717674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:43.717674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:43.799554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:43.819649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec3272220b564990b9f62896bcd4e158'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.829054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec3272220b564990b9f62896bcd4e158'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.829930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec3272220b564990b9f62896bcd4e158'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.830841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec3272220b564990b9f62896bcd4e158'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.834689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1032/2087] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:43 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:43.839807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:43.844015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:43.851630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b62e9172-8d67-49da-b0b1-2b52c7fe0969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec3272220b564990b9f62896bcd4e158'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:43.933685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc391297-e99f-47cb-acb9-280c0272ff2e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:43.941411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:43.942302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.942764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.943302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.944527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:43.947340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:43.960541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1056/2088] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:43 2022] DELETE /identity/v3/users/ec3272220b564990b9f62896bcd4e158 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:43.964845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79ced5-f97f-4816-875c-a75ec30a47c0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:43.965067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79ced5-f97f-4816-875c-a75ec30a47c0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:43.965360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79ced5-f97f-4816-875c-a75ec30a47c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:43.999761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2d79ced5-f97f-4816-875c-a75ec30a47c0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:44.000997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2d79ced5-f97f-4816-875c-a75ec30a47c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.068178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1033/2089] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:43 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:44.069606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.077423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1057/2090] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:44.084697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfd965-f06e-4a78-99ac-74630722f5fa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:44.087306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac76bc9-aefa-4423-bd2a-5f241a1b5e13 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.087306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac76bc9-aefa-4423-bd2a-5f241a1b5e13 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.087306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac76bc9-aefa-4423-bd2a-5f241a1b5e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.099265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:44.100164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.102955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.102955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.102955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:44.103869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:44.109794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4ac76bc9-aefa-4423-bd2a-5f241a1b5e13 None None] MFA Rules not processed for user `1a800d95ac1d492bb44947fdf5bb3377`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:44.114422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4ac76bc9-aefa-4423-bd2a-5f241a1b5e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.127900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1034/2091] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:44.128960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.131129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.187881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1058/2092] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:44.188583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac76bc9-aefa-4423-bd2a-5f241a1b5e13 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.200501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac76bc9-aefa-4423-bd2a-5f241a1b5e13 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:44.205730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fe239-52d6-404c-9ffe-a7a49ddaff05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:44.206434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fe239-52d6-404c-9ffe-a7a49ddaff05 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.206785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fe239-52d6-404c-9ffe-a7a49ddaff05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.206990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fe239-52d6-404c-9ffe-a7a49ddaff05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.207115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fed52-958b-495b-8f99-fe2de8ca72fd admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:44.207840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fe239-52d6-404c-9ffe-a7a49ddaff05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:44.209537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fe239-52d6-404c-9ffe-a7a49ddaff05 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:44.211580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89f9b2-65f5-445f-a92f-8dda37024129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:44.212209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f9b2-65f5-445f-a92f-8dda37024129 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.212496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f9b2-65f5-445f-a92f-8dda37024129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.212774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f9b2-65f5-445f-a92f-8dda37024129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.223252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1059/2093] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:44.225051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f2d69-da1a-4626-979a-7aefe29f036c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.225481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f2d69-da1a-4626-979a-7aefe29f036c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.225770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f2d69-da1a-4626-979a-7aefe29f036c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.227409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1060/2094] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:22:44.228176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f2d69-da1a-4626-979a-7aefe29f036c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.228386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89f9b2-65f5-445f-a92f-8dda37024129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:44.230200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89f9b2-65f5-445f-a92f-8dda37024129 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:44.243545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f2d69-da1a-4626-979a-7aefe29f036c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:44.248941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:44.249884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.250168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.250452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] PATH_INFO: `/v3/projects/f51719cf13a04e2aab8f4212657db8dd/users/44e48136c08741d4ae95d7bf42da24c5/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.251553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1035/2095] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:44.253116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.253769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.254122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.284752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] MFA Rules not processed for user `e68c847a5a60461a840679bbefc8ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:44.292803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.293432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51719cf13a04e2aab8f4212657db8dd, user_id=44e48136c08741d4ae95d7bf42da24c5, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:44.293432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:44.362185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1036/2096] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:22:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:44.364972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.366359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.380678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df6d4c-116b-4c85-bcef-8044444cb05d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:44.386289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:44.387026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.387344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.387621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] PATH_INFO: `/v3/users/306ea8d439d94da18d43d91ce9838e19` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.388799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306ea8d439d94da18d43d91ce9838e19)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:44.390723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:44.469798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1061/2097] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:44 2022] PUT /identity/v3/projects/f51719cf13a04e2aab8f4212657db8dd/users/44e48136c08741d4ae95d7bf42da24c5/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:44.471316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.473475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.543883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4728a-47d7-4838-b19f-77d8ed6284be admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:44.549024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32a7ed-fd9e-4d16-8997-6ac1fea7f8f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:44.549655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32a7ed-fd9e-4d16-8997-6ac1fea7f8f5 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.549940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32a7ed-fd9e-4d16-8997-6ac1fea7f8f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.550219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32a7ed-fd9e-4d16-8997-6ac1fea7f8f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.559041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32a7ed-fd9e-4d16-8997-6ac1fea7f8f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:44.560674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32a7ed-fd9e-4d16-8997-6ac1fea7f8f5 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:44.570991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1062/2098] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:44.572289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a65c6-4f51-40bd-8fba-7f2fd8b9e5f9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.572581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a65c6-4f51-40bd-8fba-7f2fd8b9e5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.572863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a65c6-4f51-40bd-8fba-7f2fd8b9e5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.602985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d82a65c6-4f51-40bd-8fba-7f2fd8b9e5f9 None None] MFA Rules not processed for user `44e48136c08741d4ae95d7bf42da24c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:44.604246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d82a65c6-4f51-40bd-8fba-7f2fd8b9e5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.678387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1063/2099] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:44.683065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a65c6-4f51-40bd-8fba-7f2fd8b9e5f9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.717210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306ea8d439d94da18d43d91ce9838e19'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:44.717847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306ea8d439d94da18d43d91ce9838e19'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:44.725390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306ea8d439d94da18d43d91ce9838e19'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:44.765544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a65c6-4f51-40bd-8fba-7f2fd8b9e5f9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:44.770784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:44.771443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:44.771733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:44.772019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:44.772698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:44.774640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:44.885295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306ea8d439d94da18d43d91ce9838e19'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:44.900564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306ea8d439d94da18d43d91ce9838e19'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:44.912362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1037/2100] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:44 2022] DELETE /identity/v3/users/306ea8d439d94da18d43d91ce9838e19 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:44.943614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1064/2101] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:44 2022] POST /identity/v3/projects => generated 379 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:44.948267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:44.956100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:44.989171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:45.076117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93434-d00e-4b83-bba1-b8678dfee686 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:45.086757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:45.160064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:45.160710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.160996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.161335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.162238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:45.164344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:45.184224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:45.186385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:45.200803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b70418-e8b8-4225-9753-bcc749b65d9e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:45.206411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:45.207121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.207455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.209488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] PATH_INFO: `/v3/projects/45e265d81c4d4cac9cb32ed1d95516f8` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.209488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e265d81c4d4cac9cb32ed1d95516f8)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:45.210915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:45.270091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1038/2102] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:44 2022] POST /identity/v3/users => generated 290 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:45.274592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:45.285308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966a18-d90d-48c3-9176-fd4c27b425d1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:45.290090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:45.290723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.291011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.291315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.292028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:45.293661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:45.308124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1039/2103] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:45.308124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:45.308124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:45.321079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f7ef4-5124-43a6-827c-b45b7726721d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:45.326345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:45.327095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.327424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.330164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] PATH_INFO: `/v3/projects/74bbbe86d44348c0b1ed8ff50f493e28` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.330164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74bbbe86d44348c0b1ed8ff50f493e28)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:45.331156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:45.344938 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e265d81c4d4cac9cb32ed1d95516f8', 'request_id': 'req-a4ecc60a-39f1-4df9-bb43-1f100783a617'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.485378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e265d81c4d4cac9cb32ed1d95516f8', 'request_id': 'req-a4ecc60a-39f1-4df9-bb43-1f100783a617'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.492485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e265d81c4d4cac9cb32ed1d95516f8', 'request_id': 'req-a4ecc60a-39f1-4df9-bb43-1f100783a617'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.503555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] The token cache is being invalidate because project 45e265d81c4d4cac9cb32ed1d95516f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:45.503846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.504497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.506042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1065/2104] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:45 2022] DELETE /identity/v3/projects/45e265d81c4d4cac9cb32ed1d95516f8 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:45.507008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:45.509909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:45.545180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74bbbe86d44348c0b1ed8ff50f493e28', 'request_id': 'req-f55aa372-b115-465f-af46-69f42f703345'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.553210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74bbbe86d44348c0b1ed8ff50f493e28', 'request_id': 'req-f55aa372-b115-465f-af46-69f42f703345'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.578674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc60a-39f1-4df9-bb43-1f100783a617 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:45.639221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:45.639889 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.640176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.640452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.641155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:45.642829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:45.656374 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1066/2105] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:45.658109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:45.659860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:45.673214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7b33-c67a-4c1a-a269-0c139caf7aa7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:45.678621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74bbbe86d44348c0b1ed8ff50f493e28', 'request_id': 'req-f55aa372-b115-465f-af46-69f42f703345'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.678752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:45.679284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.679571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.679857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] PATH_INFO: `/v3/users/52818dc9985d4d7e94ea98674f285aa4` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.681155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=52818dc9985d4d7e94ea98674f285aa4)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:45.683020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:45.689184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] The token cache is being invalidate because project 74bbbe86d44348c0b1ed8ff50f493e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:45.689554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.690991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.692870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1040/2106] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:45 2022] DELETE /identity/v3/projects/74bbbe86d44348c0b1ed8ff50f493e28 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:45.693733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:45.695261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:45.714032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55aa372-b115-465f-af46-69f42f703345 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:45.719436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90327178-2b07-469b-9b42-2e5cf8429660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:45.720620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327178-2b07-469b-9b42-2e5cf8429660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.720977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327178-2b07-469b-9b42-2e5cf8429660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.721323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327178-2b07-469b-9b42-2e5cf8429660 admin admin] PATH_INFO: `/v3/projects/9a8c86794b0c40d88f3722cb0c6b7e6d/users/d1dfb0d34dd54e138d94d09de0b5c303/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.744300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90327178-2b07-469b-9b42-2e5cf8429660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8c86794b0c40d88f3722cb0c6b7e6d, user_id=d1dfb0d34dd54e138d94d09de0b5c303, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:45.747111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90327178-2b07-469b-9b42-2e5cf8429660 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:45.874172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1041/2107] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:45 2022] PUT /identity/v3/projects/9a8c86794b0c40d88f3722cb0c6b7e6d/users/d1dfb0d34dd54e138d94d09de0b5c303/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:45.875806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eb2c4-77c7-4f10-9dbd-42b116bbe8f8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.876760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eb2c4-77c7-4f10-9dbd-42b116bbe8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.877197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eb2c4-77c7-4f10-9dbd-42b116bbe8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.900023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-741eb2c4-77c7-4f10-9dbd-42b116bbe8f8 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:45.901336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-741eb2c4-77c7-4f10-9dbd-42b116bbe8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:45.930545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52818dc9985d4d7e94ea98674f285aa4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.938098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52818dc9985d4d7e94ea98674f285aa4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.938926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52818dc9985d4d7e94ea98674f285aa4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.939638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52818dc9985d4d7e94ea98674f285aa4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.952012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a618b4db-3d47-414a-8f86-68b1ac289eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52818dc9985d4d7e94ea98674f285aa4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:45.966282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1042/2108] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:45.967572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:45.967860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:45.968138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:45.996584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] MFA Rules not processed for user `d1dfb0d34dd54e138d94d09de0b5c303`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:45.997809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:46.072453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1067/2109] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:45 2022] DELETE /identity/v3/users/52818dc9985d4d7e94ea98674f285aa4 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:46.074110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f73f4-91ec-4d37-85c1-3f22115182bf None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.074647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f73f4-91ec-4d37-85c1-3f22115182bf None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.075032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f73f4-91ec-4d37-85c1-3f22115182bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.084573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1043/2110] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:46.092152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.094338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:46.105593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-625f73f4-91ec-4d37-85c1-3f22115182bf None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:46.109121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-625f73f4-91ec-4d37-85c1-3f22115182bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:46.115216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e61e8-6646-4356-bbaf-81348dd8d4e0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.169652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:46.189429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.189429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.190454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1068/2111] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:46.190696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.190696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.190696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.193022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.193768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f73f4-91ec-4d37-85c1-3f22115182bf None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.206799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f73f4-91ec-4d37-85c1-3f22115182bf None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.225180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.225180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.225180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.225180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.225735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:46.225735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.225735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.237544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1069/2112] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:22:46.243565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.254818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a6472-0567-4dea-8493-a7f739f274dd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.260468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.261172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.261450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.261782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.262394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.264462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.327679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1044/2113] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:46 2022] POST /identity/v3/projects => generated 377 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:46.332359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.395006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a3f5-8523-4915-acfd-d6db0fbf23a3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.399870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.400519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.400790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.401068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.401742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.403926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.430046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1070/2114] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:46 2022] POST /identity/v3/projects => generated 393 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:46.437072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.507227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fa23e-ec10-4d86-9d92-f4f12a724df5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.512246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.513051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.513411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.513744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.514530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.516693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.539250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1045/2115] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:46 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:46.543223 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.556009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d0594-b54d-4ca2-9684-d3da23efc337 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.562521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.566310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.567297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.567908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.570116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:46.570626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.572515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.586566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1046/2116] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:46.591889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.606636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9c925-691e-4021-a642-6cd3ddfc867a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.613364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.614514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.615061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.615522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] PATH_INFO: `/v3/projects/d3a1f8651f8b4266908995324ffc72e5/users/ba352784be344006a565c56c17fa1211/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.630371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1071/2117] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:46 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:22:46.634445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.639211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a1f8651f8b4266908995324ffc72e5, user_id=ba352784be344006a565c56c17fa1211, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.639211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.645790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83aea1-c20e-4bac-b79f-ca4c85b11128 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.650614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.651339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.651688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.652030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.652789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.654493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.676772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1072/2118] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:46.681065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.692509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0887649-4497-45b8-a32b-f718ab2d627a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.697238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.697865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.698151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.698429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.699126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.700841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.719270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1073/2119] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:46.721961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.742565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f69bb9-10ef-45cd-af0b-554d3614b7ed admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.749593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.750398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.751147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.752136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] PATH_INFO: `/v3/projects/fe6c2dfb82174a5692a746ea51769a94/users/9e50546af342492b9b6f6127b0ce2c61/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.757598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1047/2120] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:46 2022] PUT /identity/v3/projects/d3a1f8651f8b4266908995324ffc72e5/users/ba352784be344006a565c56c17fa1211/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:46.760188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.762589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:46.768284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6c2dfb82174a5692a746ea51769a94, user_id=9e50546af342492b9b6f6127b0ce2c61, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.771225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.827965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dacfd6-c5a0-402b-a764-a3b7613e8c2f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.834044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.834900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.835387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.835868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] PATH_INFO: `/v3/projects/0e2008842f754b13bec9a4d029f588f4` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.837125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2008842f754b13bec9a4d029f588f4)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.839057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.849926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1074/2121] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:46 2022] PUT /identity/v3/projects/fe6c2dfb82174a5692a746ea51769a94/users/9e50546af342492b9b6f6127b0ce2c61/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:22:46.851519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:46.930984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230ad35-ead0-4924-aa96-3e2de1941cb7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:46.938535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446d89d-c154-43e1-ad06-247d65c2256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:46.939947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446d89d-c154-43e1-ad06-247d65c2256e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.940369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446d89d-c154-43e1-ad06-247d65c2256e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.940823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446d89d-c154-43e1-ad06-247d65c2256e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.941870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446d89d-c154-43e1-ad06-247d65c2256e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:46.944182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446d89d-c154-43e1-ad06-247d65c2256e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:46.955591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1075/2122] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:46.961757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f4bfc-0103-42f3-8746-5000022fc80c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:46.961757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f4bfc-0103-42f3-8746-5000022fc80c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:46.961757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f4bfc-0103-42f3-8746-5000022fc80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:46.973411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2008842f754b13bec9a4d029f588f4', 'request_id': 'req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:46.979473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d37f4bfc-0103-42f3-8746-5000022fc80c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:46.984796 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2008842f754b13bec9a4d029f588f4', 'request_id': 'req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:46.984926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d37f4bfc-0103-42f3-8746-5000022fc80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.045018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1076/2123] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:47.049888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f4bfc-0103-42f3-8746-5000022fc80c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.060232 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2008842f754b13bec9a4d029f588f4', 'request_id': 'req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:47.061463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f4bfc-0103-42f3-8746-5000022fc80c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.066400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58f0e4-dd1c-4bf9-8ecf-ba6b0b5c05ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.067625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f0e4-dd1c-4bf9-8ecf-ba6b0b5c05ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.068006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f0e4-dd1c-4bf9-8ecf-ba6b0b5c05ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.068527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f0e4-dd1c-4bf9-8ecf-ba6b0b5c05ec admin admin] PATH_INFO: `/v3/projects/d3a1f8651f8b4266908995324ffc72e5/users/ba352784be344006a565c56c17fa1211/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.070765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58f0e4-dd1c-4bf9-8ecf-ba6b0b5c05ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a1f8651f8b4266908995324ffc72e5, user_id=ba352784be344006a565c56c17fa1211, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.073691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58f0e4-dd1c-4bf9-8ecf-ba6b0b5c05ec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.073895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] The token cache is being invalidate because project 0e2008842f754b13bec9a4d029f588f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:47.074368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:47.075323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:47.077055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1048/2124] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:46 2022] DELETE /identity/v3/projects/0e2008842f754b13bec9a4d029f588f4 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:47.077989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.079393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.091363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1077/2125] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:47 2022] PUT /identity/v3/projects/d3a1f8651f8b4266908995324ffc72e5/users/ba352784be344006a565c56c17fa1211/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:47.093695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c7a9-1358-47df-89a2-b74a31edd252 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.094199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c7a9-1358-47df-89a2-b74a31edd252 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.094553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43c7a9-1358-47df-89a2-b74a31edd252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.120642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9f43c7a9-1358-47df-89a2-b74a31edd252 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:47.122158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9f43c7a9-1358-47df-89a2-b74a31edd252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.149791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fb315-2acd-4e4a-b56e-e4097d3251f6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.198281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1078/2126] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:47.198281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad8fa5-11fd-4afc-b8e3-ea96763ab4b3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.198281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad8fa5-11fd-4afc-b8e3-ea96763ab4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.198281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad8fa5-11fd-4afc-b8e3-ea96763ab4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.207157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a3780-0676-4460-b02b-9e184f379512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.207856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a3780-0676-4460-b02b-9e184f379512 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.208145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a3780-0676-4460-b02b-9e184f379512 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.208430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a3780-0676-4460-b02b-9e184f379512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.216008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1b1a3780-0676-4460-b02b-9e184f379512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.223580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a3780-0676-4460-b02b-9e184f379512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.225055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a3780-0676-4460-b02b-9e184f379512 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.228474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-60ad8fa5-11fd-4afc-b8e3-ea96763ab4b3 None None] MFA Rules not processed for user `ba352784be344006a565c56c17fa1211`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:47.229693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-60ad8fa5-11fd-4afc-b8e3-ea96763ab4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.237351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1049/2127] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:47.238670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7748a-25e5-4f9e-9420-3d28c93a6e9c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.243450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7748a-25e5-4f9e-9420-3d28c93a6e9c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.243450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7748a-25e5-4f9e-9420-3d28c93a6e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.264814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cec7748a-25e5-4f9e-9420-3d28c93a6e9c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:47.266527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cec7748a-25e5-4f9e-9420-3d28c93a6e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.276986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1050/2128] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:47.281398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7748a-25e5-4f9e-9420-3d28c93a6e9c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.291892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7748a-25e5-4f9e-9420-3d28c93a6e9c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.296762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.297437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.297716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.298161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.299287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:22:47.299659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.301487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.310532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1079/2129] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:47.319351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1051/2130] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:22:47.326382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad8fa5-11fd-4afc-b8e3-ea96763ab4b3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.330150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.337923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.342334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad8fa5-11fd-4afc-b8e3-ea96763ab4b3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.353111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.353749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.354061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.354309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.354985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.356996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.409640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f20b2-f292-4589-9c41-426a4bf426fa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.413421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.414083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.414367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.414772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.424720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.426194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.436593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1052/2131] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:47.487708 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1080/2132] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:22:47 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:47.499596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.510413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c471e-480c-4a2f-8932-e23228cd7541 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.570415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.571035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.571353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.571756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.572452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.574401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.606041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:47.639812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.730153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1053/2133] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:22:47 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:22:47.731982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.744272 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383241d-1bce-4b78-95a4-4322c56b1557 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.750175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa73e-1d1c-4aa7-a958-6523163e4a30 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.754064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.755016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.755433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.755556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d430139-d438-42c3-ad67-ba539579e242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.755922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.759295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d430139-d438-42c3-ad67-ba539579e242 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.759295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d430139-d438-42c3-ad67-ba539579e242 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.759295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d430139-d438-42c3-ad67-ba539579e242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.760184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.760184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.760674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7d430139-d438-42c3-ad67-ba539579e242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:47.773600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1054/2134] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:22:47.778424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.799437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e60d6-a8f5-4045-ae0e-07669dd59652 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.803875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.804501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.804862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.805214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.806213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.808163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.823380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1055/2135] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:22:47.828082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:47.841755 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa37155-4bdf-44d3-8b17-5fe60737268b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:47.848599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:47.848868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:47.849306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:47.849696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] PATH_INFO: `/v3/projects/2874a8802e804b56ac7727cd9369031c/users/ac0cdf901468494aaa76b43ccb01f848/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:47.881236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2874a8802e804b56ac7727cd9369031c, user_id=ac0cdf901468494aaa76b43ccb01f848, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.895134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.912190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d430139-d438-42c3-ad67-ba539579e242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:47.915331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d430139-d438-42c3-ad67-ba539579e242 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:47.924604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:47.935172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1081/2136] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:22:48.004081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1056/2137] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:22:47 2022] PUT /identity/v3/projects/2874a8802e804b56ac7727cd9369031c/users/ac0cdf901468494aaa76b43ccb01f848/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:48.016420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18e05f-566b-4ecb-a450-7f2b62503eae None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:48.016728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18e05f-566b-4ecb-a450-7f2b62503eae None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:48.017548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18e05f-566b-4ecb-a450-7f2b62503eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:48.068991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ea18e05f-566b-4ecb-a450-7f2b62503eae None None] MFA Rules not processed for user `ac0cdf901468494aaa76b43ccb01f848`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:48.070489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ea18e05f-566b-4ecb-a450-7f2b62503eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:48.190150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1082/2138] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:48.526519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:48.533858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:48.635615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2629b8-a959-4c61-8543-5026c9373736 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:48.640994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f85495-c5e2-42ff-a4d6-a127481e856b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:48.641644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85495-c5e2-42ff-a4d6-a127481e856b service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:48.641971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85495-c5e2-42ff-a4d6-a127481e856b service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:48.642251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85495-c5e2-42ff-a4d6-a127481e856b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:48.652530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f85495-c5e2-42ff-a4d6-a127481e856b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:48.664291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f85495-c5e2-42ff-a4d6-a127481e856b service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:48.665744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1057/2139] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:49.303280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:50.175264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:52.138553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:52.229918 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:54.058504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:54.219207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:54.219670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:54.219836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:54.225661 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:54.259322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] MFA Rules not processed for user `9e50546af342492b9b6f6127b0ce2c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:54.260574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:54.342004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1083/2140] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:54.358691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c5dec-54cf-43a2-a4df-9d17fa780549 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:54.358691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c5dec-54cf-43a2-a4df-9d17fa780549 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:54.358691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c5dec-54cf-43a2-a4df-9d17fa780549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:54.376939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-423c5dec-54cf-43a2-a4df-9d17fa780549 None None] MFA Rules not processed for user `9e50546af342492b9b6f6127b0ce2c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:54.379293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-423c5dec-54cf-43a2-a4df-9d17fa780549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:54.389957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1058/2141] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:54.403128 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:54.405485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:54.489585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258ff41-3e6f-4a2e-8286-720058e04f89 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:54.495324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:54.497612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:54.497612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:54.497612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:54.505475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:54.507221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:54.517424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1084/2142] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:55.452342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98ffe-8098-45ae-a9af-15f4d3ccf19b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:55.452342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98ffe-8098-45ae-a9af-15f4d3ccf19b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:55.452342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98ffe-8098-45ae-a9af-15f4d3ccf19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:55.469241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8ba98ffe-8098-45ae-a9af-15f4d3ccf19b None None] MFA Rules not processed for user `9e50546af342492b9b6f6127b0ce2c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:55.470493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8ba98ffe-8098-45ae-a9af-15f4d3ccf19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:55.479286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1059/2143] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:22:55.497541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:55.954742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:57.646580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:57.896143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:57.898146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:57.972013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e332d4-de27-4d26-b895-72ddda5b26d7 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:57.977347 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:57.978212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:57.978464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:57.979465 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] PATH_INFO: `/v3/users/44e48136c08741d4ae95d7bf42da24c5` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:57.981893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e48136c08741d4ae95d7bf42da24c5)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:57.986147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:58.005753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:58.213797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e48136c08741d4ae95d7bf42da24c5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:58.226315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e48136c08741d4ae95d7bf42da24c5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:58.227695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e48136c08741d4ae95d7bf42da24c5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:58.230941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e48136c08741d4ae95d7bf42da24c5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:58.245100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e48136c08741d4ae95d7bf42da24c5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:58.367984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1085/2144] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:57 2022] DELETE /identity/v3/users/44e48136c08741d4ae95d7bf42da24c5 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:58.372447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:58.372748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:58.373063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:58.392579 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:22:58.393879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:58.459898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1060/2145] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:22:58.470552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:58.472000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:58.528652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab4c5a-2db0-4c3d-9057-7c22038ee325 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:58.537093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:58.537093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:58.537265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:58.538100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:58.551237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:58.553596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:58.566766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1086/2146] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:22:59.351078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:59.353314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:59.395378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750a324-1ded-494d-b707-d7399a3d9c82 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:59.410120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:59.411033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:59.411502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:59.411976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] PATH_INFO: `/v3/projects/f51719cf13a04e2aab8f4212657db8dd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:59.413787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f51719cf13a04e2aab8f4212657db8dd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:59.420156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:59.453834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:59.604401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f51719cf13a04e2aab8f4212657db8dd', 'request_id': 'req-bda3f19e-29e7-4770-b331-c269dcc0d117'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:59.616921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f51719cf13a04e2aab8f4212657db8dd', 'request_id': 'req-bda3f19e-29e7-4770-b331-c269dcc0d117'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:59.686524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f51719cf13a04e2aab8f4212657db8dd', 'request_id': 'req-bda3f19e-29e7-4770-b331-c269dcc0d117'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:59.701678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] The token cache is being invalidate because project f51719cf13a04e2aab8f4212657db8dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:22:59.702258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:59.703802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:22:59.707370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1061/2147] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:22:59 2022] DELETE /identity/v3/projects/f51719cf13a04e2aab8f4212657db8dd => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:22:59.711037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:59.721947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c477c-41ae-494f-a9eb-34529d04434b service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:59.787448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:59.788104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:59.788390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:59.788671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] PATH_INFO: `/v3/users/d1dfb0d34dd54e138d94d09de0b5c303` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:59.789844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1dfb0d34dd54e138d94d09de0b5c303)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:59.791827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:00.055360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1dfb0d34dd54e138d94d09de0b5c303'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:00.065068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1dfb0d34dd54e138d94d09de0b5c303'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:00.067018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1dfb0d34dd54e138d94d09de0b5c303'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:00.067018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1dfb0d34dd54e138d94d09de0b5c303'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:00.082203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1dfb0d34dd54e138d94d09de0b5c303'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:00.213023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1087/2148] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:22:59 2022] DELETE /identity/v3/users/d1dfb0d34dd54e138d94d09de0b5c303 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:00.539631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:00.726971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:00.728491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:00.748563 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3f19e-29e7-4770-b331-c269dcc0d117 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:00.770637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:00.774096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:00.855015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:00.855822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:00.856248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:00.856627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] PATH_INFO: `/v3/users/0446afe44d5c4a518d35c5520d3e869a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:00.858092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0446afe44d5c4a518d35c5520d3e869a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:00.861292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:00.923335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e6f4b-b006-4078-9ffb-0742a6749b52 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:00.929330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:00.930737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:00.931073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:00.931480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] PATH_INFO: `/v3/projects/9a8c86794b0c40d88f3722cb0c6b7e6d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:00.932717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a8c86794b0c40d88f3722cb0c6b7e6d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:00.953702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:01.120876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0446afe44d5c4a518d35c5520d3e869a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.127528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0446afe44d5c4a518d35c5520d3e869a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.141640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0446afe44d5c4a518d35c5520d3e869a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.151769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a8c86794b0c40d88f3722cb0c6b7e6d', 'request_id': 'req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.257226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0446afe44d5c4a518d35c5520d3e869a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.271730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33c441a5-1092-44d5-b9af-94f4f65c5fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0446afe44d5c4a518d35c5520d3e869a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.283699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1062/2149] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:00 2022] DELETE /identity/v3/users/0446afe44d5c4a518d35c5520d3e869a => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:01.285408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:01.285783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:01.286142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:01.315312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:01.316066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:01.318074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:01.338193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a8c86794b0c40d88f3722cb0c6b7e6d', 'request_id': 'req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.349826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a8c86794b0c40d88f3722cb0c6b7e6d', 'request_id': 'req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.375661 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] The token cache is being invalidate because project 9a8c86794b0c40d88f3722cb0c6b7e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:01.376251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.377394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.380603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1088/2150] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:00 2022] DELETE /identity/v3/projects/9a8c86794b0c40d88f3722cb0c6b7e6d => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:01.384803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:01.387299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:01.446541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1063/2151] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:01.467146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:01.469531 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:01.489338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb511e-e2f8-4dbc-846d-e2cb2e6c917e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:01.497301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:01.498247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:01.498764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:01.499321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] PATH_INFO: `/v3/users/ba352784be344006a565c56c17fa1211` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:01.500823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba352784be344006a565c56c17fa1211)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:01.503955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:01.575615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1d832-f071-49b9-87ba-59f06f1cd8cd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:01.580140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:01.580775 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:01.581068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:01.581413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:01.597442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:01.598939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:01.614650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1064/2152] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:01.614650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:01.614650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:01.638846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848805d-8407-4248-90ad-d208c3d6e900 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:01.649837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:01.650505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:01.650926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:01.651235 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] PATH_INFO: `/v3/users/9ae724b52eb24159b88595317e3d5aa1` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:01.653090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae724b52eb24159b88595317e3d5aa1)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:01.655999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:01.836424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba352784be344006a565c56c17fa1211'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.868271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba352784be344006a565c56c17fa1211'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.871655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba352784be344006a565c56c17fa1211'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.877950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba352784be344006a565c56c17fa1211'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.907968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae724b52eb24159b88595317e3d5aa1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.909748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae724b52eb24159b88595317e3d5aa1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.920950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae724b52eb24159b88595317e3d5aa1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:01.948965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba352784be344006a565c56c17fa1211'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.028961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae724b52eb24159b88595317e3d5aa1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.045259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc41e51b-8b92-42c2-b240-cde55cd17a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae724b52eb24159b88595317e3d5aa1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.069074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1065/2153] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:01 2022] DELETE /identity/v3/users/9ae724b52eb24159b88595317e3d5aa1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:02.073501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:02.074202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:02.074807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:02.105478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:02.107391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:02.144633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1089/2154] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:01 2022] DELETE /identity/v3/users/ba352784be344006a565c56c17fa1211 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:02.208585 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1066/2155] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:02.216793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:02.218494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:02.308249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eaaff-b58d-4467-9d0a-ee76f1ed2e58 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:02.312538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:02.313643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:02.314103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:02.319464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:02.341604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:02.343396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:02.370515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cc609-83b7-4c43-868c-794c51b03eb9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:02.446062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:02.448363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:02.463503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1090/2156] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:02.484217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:02.484946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:02.485948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:02.486382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] PATH_INFO: `/v3/projects/82920a312dfb4338872c376816c742a3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:02.488318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82920a312dfb4338872c376816c742a3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:02.490622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:02.661743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82920a312dfb4338872c376816c742a3', 'request_id': 'req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.672342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82920a312dfb4338872c376816c742a3', 'request_id': 'req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.760323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82920a312dfb4338872c376816c742a3', 'request_id': 'req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.776302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] The token cache is being invalidate because project 82920a312dfb4338872c376816c742a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:02.776602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.777309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:02.778966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1067/2157] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:02 2022] DELETE /identity/v3/projects/82920a312dfb4338872c376816c742a3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:02.783728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:02.799133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff687cb-c293-4fac-b121-38fc49887a5c service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:02.874345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:02.876803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:02.878250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:02.878560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] PATH_INFO: `/v3/users/c29e2d70f5604d1dbcea3839540c9555` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:02.880305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29e2d70f5604d1dbcea3839540c9555)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:02.884190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:02.961813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:02.963573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:02.976846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:02.996417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd4788-f2e1-4c5b-9a18-ea6a8747fb23 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:03.008846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:03.008846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.008846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.008846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] PATH_INFO: `/v3/projects/d3a1f8651f8b4266908995324ffc72e5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.009400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a1f8651f8b4266908995324ffc72e5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:03.010727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:03.105466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29e2d70f5604d1dbcea3839540c9555'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.114800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29e2d70f5604d1dbcea3839540c9555'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.115540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29e2d70f5604d1dbcea3839540c9555'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.116356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29e2d70f5604d1dbcea3839540c9555'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.132130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29e2d70f5604d1dbcea3839540c9555'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.140219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a1f8651f8b4266908995324ffc72e5', 'request_id': 'req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.149372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a1f8651f8b4266908995324ffc72e5', 'request_id': 'req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.223872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1091/2158] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:02 2022] DELETE /identity/v3/users/c29e2d70f5604d1dbcea3839540c9555 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:03.226893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:03.226893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:03.292692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14fe06-6b60-40c2-b820-1834d5faf2e5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:03.297891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:03.299061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.299962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.300264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] PATH_INFO: `/v3/projects/d1915bea3ac94b3ba6529eb23ea52c2e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.301906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1915bea3ac94b3ba6529eb23ea52c2e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:03.303886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:03.342918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a1f8651f8b4266908995324ffc72e5', 'request_id': 'req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.354570 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] The token cache is being invalidate because project d3a1f8651f8b4266908995324ffc72e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:03.354866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.355524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.357055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1068/2159] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:02 2022] DELETE /identity/v3/projects/d3a1f8651f8b4266908995324ffc72e5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:03.358251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:03.359599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:03.426884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5f552-01e4-45cb-a9d4-8afd3fc656aa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:03.431661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:03.432348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.432637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.432950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] PATH_INFO: `/v3/users/ac0cdf901468494aaa76b43ccb01f848` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.434266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0cdf901468494aaa76b43ccb01f848)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:03.436255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:03.480447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1915bea3ac94b3ba6529eb23ea52c2e', 'request_id': 'req-537d231e-e938-41f1-b133-bbc1d123d34d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.603829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1915bea3ac94b3ba6529eb23ea52c2e', 'request_id': 'req-537d231e-e938-41f1-b133-bbc1d123d34d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.612844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1915bea3ac94b3ba6529eb23ea52c2e', 'request_id': 'req-537d231e-e938-41f1-b133-bbc1d123d34d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.624358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] The token cache is being invalidate because project d1915bea3ac94b3ba6529eb23ea52c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:03.624681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.625369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-537d231e-e938-41f1-b133-bbc1d123d34d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.627228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1092/2160] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:03 2022] DELETE /identity/v3/projects/d1915bea3ac94b3ba6529eb23ea52c2e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:03.628356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f49be7-56a7-46fe-afd5-4c50bf14b9e3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.628647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f49be7-56a7-46fe-afd5-4c50bf14b9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.628925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f49be7-56a7-46fe-afd5-4c50bf14b9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.651750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-98f49be7-56a7-46fe-afd5-4c50bf14b9e3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:03.653016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-98f49be7-56a7-46fe-afd5-4c50bf14b9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:03.726781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1093/2161] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:03.728531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076788ef-a395-49a6-bd09-cb425257eaa6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.728531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076788ef-a395-49a6-bd09-cb425257eaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.729126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076788ef-a395-49a6-bd09-cb425257eaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.735299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0cdf901468494aaa76b43ccb01f848'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.737354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0cdf901468494aaa76b43ccb01f848'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.750031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0cdf901468494aaa76b43ccb01f848'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.752559 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-076788ef-a395-49a6-bd09-cb425257eaa6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:03.755248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-076788ef-a395-49a6-bd09-cb425257eaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:03.767496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1094/2162] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:03.768693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1dd61-eb4a-421a-a43f-091742af6972 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.769075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1dd61-eb4a-421a-a43f-091742af6972 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.769435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1dd61-eb4a-421a-a43f-091742af6972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.793676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3cd1dd61-eb4a-421a-a43f-091742af6972 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:03.795408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3cd1dd61-eb4a-421a-a43f-091742af6972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:03.815344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1095/2163] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:03.815344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12092f89-08bf-4513-a63a-93eedc961046 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.815344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12092f89-08bf-4513-a63a-93eedc961046 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.815344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12092f89-08bf-4513-a63a-93eedc961046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.815699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0cdf901468494aaa76b43ccb01f848'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.833303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-12092f89-08bf-4513-a63a-93eedc961046 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:03.834869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-12092f89-08bf-4513-a63a-93eedc961046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:03.837796 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0cdf901468494aaa76b43ccb01f848'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:03.861617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1069/2164] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:03 2022] DELETE /identity/v3/users/ac0cdf901468494aaa76b43ccb01f848 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:03.863955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:03.867484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1096/2165] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:03.868892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f522d-637c-43da-826b-184333df749a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:03.869189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f522d-637c-43da-826b-184333df749a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:03.869534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f522d-637c-43da-826b-184333df749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:03.891604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-695f522d-637c-43da-826b-184333df749a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:03.893364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-695f522d-637c-43da-826b-184333df749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:03.987501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1097/2166] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:03.988491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f522d-637c-43da-826b-184333df749a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.008371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f522d-637c-43da-826b-184333df749a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.013911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.014542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.014833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.015146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.015889 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:04.021785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.021785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.032442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8d9a-2fd2-4f2a-8a77-ae0164b4335e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.034523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1098/2167] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:04.035574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.037873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:04.042294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.043345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.043829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.044467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.050765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:04.050765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.050765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.053385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37e88e-6210-41e6-a23e-1da6fd89e5e5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.058544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.059300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.059650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.060042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] PATH_INFO: `/v3/projects/967d7ca5b5f8499da4229d6e489fca4a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.061418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=967d7ca5b5f8499da4229d6e489fca4a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.064371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.064571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1070/2168] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:04.065455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.067577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:04.127849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd82c5f-1fe7-4e6d-90be-a1b5cf78590e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.131662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.132295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.133116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.133116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.145125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.147255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.157696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1071/2169] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:04.162917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.186682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf1f29-6bf2-4666-bf81-2e3c4f60cc7d service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.187947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967d7ca5b5f8499da4229d6e489fca4a', 'request_id': 'req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:04.274134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967d7ca5b5f8499da4229d6e489fca4a', 'request_id': 'req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:04.291700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967d7ca5b5f8499da4229d6e489fca4a', 'request_id': 'req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:04.295766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2879a7-2521-45d3-9187-552df93b659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.296459 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2879a7-2521-45d3-9187-552df93b659c admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.297054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2879a7-2521-45d3-9187-552df93b659c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.297503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2879a7-2521-45d3-9187-552df93b659c admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.298244 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2879a7-2521-45d3-9187-552df93b659c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.301292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2879a7-2521-45d3-9187-552df93b659c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.309503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] The token cache is being invalidate because project 967d7ca5b5f8499da4229d6e489fca4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:04.309801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:04.310522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:04.312467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1099/2170] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:04 2022] DELETE /identity/v3/projects/967d7ca5b5f8499da4229d6e489fca4a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:04.316611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.316611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:04.330156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9019983-b9c1-46fa-8d36-1bed9a6a5f39 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.335436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.336078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.336360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.336640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.337689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.339246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.411949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1072/2171] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:04 2022] POST /identity/v3/projects => generated 373 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:04.411949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a7e9f-640f-47c3-ad23-80bdfd19f2c1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.412247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a7e9f-640f-47c3-ad23-80bdfd19f2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.413044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a7e9f-640f-47c3-ad23-80bdfd19f2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.434794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d72a7e9f-640f-47c3-ad23-80bdfd19f2c1 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:04.436110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d72a7e9f-640f-47c3-ad23-80bdfd19f2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:04.452323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1100/2172] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:04 2022] POST /identity/v3/projects => generated 359 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:04.452323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.456057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:04.526462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1073/2173] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:04.527987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a7e9f-640f-47c3-ad23-80bdfd19f2c1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.547152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a7e9f-640f-47c3-ad23-80bdfd19f2c1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.577278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888c7a6-0500-4055-a88c-8537ba7ae72f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.584160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2e0b2-8f18-4cf3-a613-40cfcf0a58ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.584798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2e0b2-8f18-4cf3-a613-40cfcf0a58ee admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.585090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2e0b2-8f18-4cf3-a613-40cfcf0a58ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.585371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2e0b2-8f18-4cf3-a613-40cfcf0a58ee admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.586046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2e0b2-8f18-4cf3-a613-40cfcf0a58ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.588847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2e0b2-8f18-4cf3-a613-40cfcf0a58ee admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.620055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.620771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.621221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.621535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.622383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.624689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.688117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:04.695631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1101/2174] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:04 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:04.697111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db78dc-e08a-4d8e-a53b-145b528f94be None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.697543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db78dc-e08a-4d8e-a53b-145b528f94be None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.697860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db78dc-e08a-4d8e-a53b-145b528f94be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.717206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-26db78dc-e08a-4d8e-a53b-145b528f94be None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:04.718461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-26db78dc-e08a-4d8e-a53b-145b528f94be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:04.731974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1102/2175] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:04.731974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db78dc-e08a-4d8e-a53b-145b528f94be None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.740068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db78dc-e08a-4d8e-a53b-145b528f94be None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.745483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.746420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.747398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.747398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.747797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.749416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.762736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1103/2176] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:04.763877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.775455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f08170-e4e6-4d75-ac64-814c8bbaa1c8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.783288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.784136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.784570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.784984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.785893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:04.786353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.788390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.801451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1074/2177] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:04 2022] POST /identity/v3/users => generated 280 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:04.802733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.805739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1104/2178] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:04.806868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.810428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:04.826353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb297-5833-4ed1-ad06-d38562de3b5c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.837898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.838618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.838912 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.839210 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] PATH_INFO: `/v3/projects/2874a8802e804b56ac7727cd9369031c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.842250 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2874a8802e804b56ac7727cd9369031c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.846021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.854419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779288c-395d-4c69-8a83-80f03ce77fb2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.862632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.863068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.863335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.864288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.865269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.867286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.896482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1075/2179] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:04.897738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:04.927426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e77e-5381-4a18-a252-2c70c3690fe4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:04.939754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:04.940478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:04.940753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:04.941027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:04.946785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:04.948403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:04.981203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1076/2180] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:04.981930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.000162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21354ad7-174e-4a1d-a8c0-4fc746eb1806 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.008086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.009292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.009725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.010177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.010876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.014616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.166277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2874a8802e804b56ac7727cd9369031c', 'request_id': 'req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:05.195453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1077/2181] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:04 2022] POST /identity/v3/projects => generated 387 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:05.196257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.259095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ce137-f2cd-4e60-898c-e42b96b8c94d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.264028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43087b6d-3097-4b26-8939-f0992611c289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.264644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43087b6d-3097-4b26-8939-f0992611c289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.264929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43087b6d-3097-4b26-8939-f0992611c289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.265207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43087b6d-3097-4b26-8939-f0992611c289 admin admin] PATH_INFO: `/v3/projects/3c152c0bd9bb436e8bfb41cba0a17036/users/fcecfad74acd4d398ae06ceb32317e96/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.280314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43087b6d-3097-4b26-8939-f0992611c289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c152c0bd9bb436e8bfb41cba0a17036, user_id=fcecfad74acd4d398ae06ceb32317e96, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.283758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43087b6d-3097-4b26-8939-f0992611c289 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.308572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2874a8802e804b56ac7727cd9369031c', 'request_id': 'req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:05.315521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2874a8802e804b56ac7727cd9369031c', 'request_id': 'req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:05.327729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] The token cache is being invalidate because project 2874a8802e804b56ac7727cd9369031c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:05.334895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:05.334895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:05.334895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1105/2182] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:04 2022] DELETE /identity/v3/projects/2874a8802e804b56ac7727cd9369031c => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:05.334895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.334895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:05.398556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b939440-8ef1-44d5-9e6c-fbe8f55acf6e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.403108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.403758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.404045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.404320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.405008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.406615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.416969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1106/2183] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:05.417753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.419421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:05.431537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701d2e-61e4-4171-b465-ed20fa4218ba admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.436144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db972da1-9adf-43ee-babd-c5ce0e2b3307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.436753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db972da1-9adf-43ee-babd-c5ce0e2b3307 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.437022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db972da1-9adf-43ee-babd-c5ce0e2b3307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.437303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db972da1-9adf-43ee-babd-c5ce0e2b3307 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.437961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db972da1-9adf-43ee-babd-c5ce0e2b3307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.439999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db972da1-9adf-43ee-babd-c5ce0e2b3307 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.471379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1078/2184] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:05 2022] PUT /identity/v3/projects/3c152c0bd9bb436e8bfb41cba0a17036/users/fcecfad74acd4d398ae06ceb32317e96/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:05.473153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82920af7-ffee-4726-86dc-048dca168144 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.473571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82920af7-ffee-4726-86dc-048dca168144 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.473993 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82920af7-ffee-4726-86dc-048dca168144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.493437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-82920af7-ffee-4726-86dc-048dca168144 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:05.494671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-82920af7-ffee-4726-86dc-048dca168144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:05.554806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1079/2185] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:05.557249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1107/2186] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:05 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:05.557384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82920af7-ffee-4726-86dc-048dca168144 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.557474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe255-fd02-44ed-a77b-7ec71ea91e38 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.557474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe255-fd02-44ed-a77b-7ec71ea91e38 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.557474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe255-fd02-44ed-a77b-7ec71ea91e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.565520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82920af7-ffee-4726-86dc-048dca168144 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.570174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.570856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.571136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.571447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] PATH_INFO: `/v3/projects/fd4bc3eee0f94562913239be525b6438/users/b23af5831e4d488ea89dce58210a2f1f/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.585956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bd0fe255-fd02-44ed-a77b-7ec71ea91e38 None None] MFA Rules not processed for user `fcecfad74acd4d398ae06ceb32317e96`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:05.587137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bd0fe255-fd02-44ed-a77b-7ec71ea91e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:05.593447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4bc3eee0f94562913239be525b6438, user_id=b23af5831e4d488ea89dce58210a2f1f, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.607393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.655981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1108/2187] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:05.660736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f236bec-d0d9-4463-a5f9-485f54048a74 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.661170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f236bec-d0d9-4463-a5f9-485f54048a74 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.661524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f236bec-d0d9-4463-a5f9-485f54048a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.681506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4f236bec-d0d9-4463-a5f9-485f54048a74 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:05.683280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4f236bec-d0d9-4463-a5f9-485f54048a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:05.696388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1109/2188] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:05.697192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f236bec-d0d9-4463-a5f9-485f54048a74 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.707142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f236bec-d0d9-4463-a5f9-485f54048a74 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.713205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.714232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.714730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.715214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.715939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.717579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.728924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1110/2189] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:05.728924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.730839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:05.743085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1080/2190] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:05 2022] PUT /identity/v3/projects/fd4bc3eee0f94562913239be525b6438/users/b23af5831e4d488ea89dce58210a2f1f/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:05.743085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.743085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:05.805847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2d7f0-b92a-4f92-a1e4-8d7d1ad26400 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.819153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c322a-9eff-46ae-94a4-066812153106 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.824083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.824707 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.824995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.825276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.825977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:05.826287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.827937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.839913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1081/2191] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:05.840813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.852576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f6c1-1501-42f7-958f-dbd6fd31eecc admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.855940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.856578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.856864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.861196 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.861297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a51b8-abe8-45cb-b28c-0876fb31c671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.861297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a51b8-abe8-45cb-b28c-0876fb31c671 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.861297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a51b8-abe8-45cb-b28c-0876fb31c671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.861297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a51b8-abe8-45cb-b28c-0876fb31c671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.861611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a51b8-abe8-45cb-b28c-0876fb31c671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.862974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a51b8-abe8-45cb-b28c-0876fb31c671 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.873956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1082/2192] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:05.875170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dae24-d68b-4dfe-aab3-377cb447d8f8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.875485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dae24-d68b-4dfe-aab3-377cb447d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.875769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dae24-d68b-4dfe-aab3-377cb447d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.904794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b03dae24-d68b-4dfe-aab3-377cb447d8f8 None None] MFA Rules not processed for user `b23af5831e4d488ea89dce58210a2f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:05.906569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b03dae24-d68b-4dfe-aab3-377cb447d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:05.917463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.919081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.928206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1111/2193] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:05.929096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.940580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64e9ff-5272-48de-8b0d-21c6a6de10f0 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.945525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.946138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.946556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.946742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:05.947449 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:05.949470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:05.978169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1083/2194] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:05.979405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dae24-d68b-4dfe-aab3-377cb447d8f8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:05.990897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dae24-d68b-4dfe-aab3-377cb447d8f8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:05.996407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:05.997083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:05.997631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:05.998586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] PATH_INFO: `/v3/projects/f0785be18f464028aaab62d4abe7f133/users/35e28a35b1154cffb33e4db5aebfc423/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.017702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0785be18f464028aaab62d4abe7f133, user_id=35e28a35b1154cffb33e4db5aebfc423, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:06.020421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:06.058081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1112/2195] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:05 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:06.058081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:06.058081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:06.117935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1084/2196] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:05 2022] PUT /identity/v3/projects/f0785be18f464028aaab62d4abe7f133/users/35e28a35b1154cffb33e4db5aebfc423/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:06.121261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:06.122610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:06.156265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0994b-b002-4a05-9e15-b07c76a13bed admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:06.215850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c045c97-c5e2-4cae-abfe-846dd06e1b45 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:06.220900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:06.221590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.221919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.222094 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.274909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:06.275705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.275917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.276205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] PATH_INFO: `/v3/users/fcecfad74acd4d398ae06ceb32317e96` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.279030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcecfad74acd4d398ae06ceb32317e96)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:06.284027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:06.319987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:06.321516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:06.340503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1085/2197] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:06.341475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:06.354259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883d60a-c9fb-411d-9d53-1426e7e44c63 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:06.359723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f49e07-cc43-4d73-89cc-dd45441ec145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:06.360333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f49e07-cc43-4d73-89cc-dd45441ec145 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.361216 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f49e07-cc43-4d73-89cc-dd45441ec145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.361622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f49e07-cc43-4d73-89cc-dd45441ec145 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.362656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f49e07-cc43-4d73-89cc-dd45441ec145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:06.364836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f49e07-cc43-4d73-89cc-dd45441ec145 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:06.515602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1086/2198] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:06 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:06.516997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.517287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.517564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.547354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] MFA Rules not processed for user `35e28a35b1154cffb33e4db5aebfc423`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:06.547581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcecfad74acd4d398ae06ceb32317e96'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.549211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:06.556090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcecfad74acd4d398ae06ceb32317e96'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.556790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcecfad74acd4d398ae06ceb32317e96'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.557446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcecfad74acd4d398ae06ceb32317e96'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.574324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcecfad74acd4d398ae06ceb32317e96'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.621931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1087/2199] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:06.622794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:06.624382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:06.639942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc7389-23bc-48a4-83d1-5679db4e84a7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:06.644647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:06.645293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.645578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.645855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] PATH_INFO: `/v3/users/b23af5831e4d488ea89dce58210a2f1f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.647877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23af5831e4d488ea89dce58210a2f1f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:06.652808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:06.699106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1113/2200] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:06 2022] DELETE /identity/v3/users/fcecfad74acd4d398ae06ceb32317e96 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:06.703062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:06.704451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:06.783178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea1555-2933-4e37-bb8c-b7733d1fbe4c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:06.790416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:06.791214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.791608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.794835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.795628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:06.797280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:06.813755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1114/2201] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:06.813755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:06.817181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:06.831135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0d6da-1081-482b-a092-51b38178bee5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:06.835988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4501b-04fb-456a-b7fc-735a1d00a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:06.836616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4501b-04fb-456a-b7fc-735a1d00a41e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.836904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4501b-04fb-456a-b7fc-735a1d00a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.837182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4501b-04fb-456a-b7fc-735a1d00a41e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.837881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4501b-04fb-456a-b7fc-735a1d00a41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:06.839848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4501b-04fb-456a-b7fc-735a1d00a41e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:06.925754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1115/2202] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:06 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:06.925754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:06.925754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:06.925754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:06.930326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23af5831e4d488ea89dce58210a2f1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.931002 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23af5831e4d488ea89dce58210a2f1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.938640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23af5831e4d488ea89dce58210a2f1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:06.940636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:06.957807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:06.959269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:07.010890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23af5831e4d488ea89dce58210a2f1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:07.027289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1116/2203] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:07.028176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.028490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23af5831e4d488ea89dce58210a2f1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:07.029647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:07.041788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1088/2204] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:06 2022] DELETE /identity/v3/users/b23af5831e4d488ea89dce58210a2f1f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:07.042611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.042762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067e51d-6fec-46b1-96ff-eb86ee7556bc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.043977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:07.115314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.116294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.116583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.116868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.121583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.121583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.133300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9e64f-5303-4b97-a929-372f54f428ce admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.137005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1117/2205] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:07.137932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.139744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:07.141396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.142602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.142947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.143302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.144083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.147139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.208779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefea118-2721-4d17-94d4-624aeb6431a1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.212965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fc9af-412b-4888-adc6-3a379ee84af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.213619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc9af-412b-4888-adc6-3a379ee84af7 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.213880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc9af-412b-4888-adc6-3a379ee84af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.214161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc9af-412b-4888-adc6-3a379ee84af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.227141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fc9af-412b-4888-adc6-3a379ee84af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.228778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fc9af-412b-4888-adc6-3a379ee84af7 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.246895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1118/2206] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:07.248214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fcfbe-c6af-4b95-adbc-90a9d9d09f4a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.248601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fcfbe-c6af-4b95-adbc-90a9d9d09f4a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.248874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fcfbe-c6af-4b95-adbc-90a9d9d09f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.260707 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1089/2207] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:07 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:07.262992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.269553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-388fcfbe-c6af-4b95-adbc-90a9d9d09f4a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:07.270687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-388fcfbe-c6af-4b95-adbc-90a9d9d09f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:07.278067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ef7a0-bb17-4076-bb28-a6eca37a2b84 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.283835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1119/2208] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:07.288888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fcfbe-c6af-4b95-adbc-90a9d9d09f4a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.290050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a917e-f2e5-40b2-9a0f-d0c5127353e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.291028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a917e-f2e5-40b2-9a0f-d0c5127353e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.291761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a917e-f2e5-40b2-9a0f-d0c5127353e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.292298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a917e-f2e5-40b2-9a0f-d0c5127353e8 admin admin] PATH_INFO: `/v3/projects/871a7bb725ba408eb421b2c94d6ccad7/users/99778ee4e714479abe49af1bc590aeb1/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.304591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fcfbe-c6af-4b95-adbc-90a9d9d09f4a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.323815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a917e-f2e5-40b2-9a0f-d0c5127353e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871a7bb725ba408eb421b2c94d6ccad7, user_id=99778ee4e714479abe49af1bc590aeb1, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.326998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.327962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a917e-f2e5-40b2-9a0f-d0c5127353e8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.328746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.330140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.330505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.333369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.336040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.370398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1120/2209] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:07.370398 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.381043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85509e1c-79c1-44f2-8d59-d016decd2de4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.385119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.385975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.386479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.387019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.398811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.400755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.416687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1121/2210] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:07.417837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.436141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9be37-e2af-4a59-9bec-a28f84606f28 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.448000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.448538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.448932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.449813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] PATH_INFO: `/v3/projects/7037772da7744e1c975187e9a7bf035c/users/b1132fb9b25c4c74a813aec6e8bf7c97/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.473702 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7037772da7744e1c975187e9a7bf035c, user_id=b1132fb9b25c4c74a813aec6e8bf7c97, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.477895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.519251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1090/2211] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:07 2022] PUT /identity/v3/projects/871a7bb725ba408eb421b2c94d6ccad7/users/99778ee4e714479abe49af1bc590aeb1/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:07.519251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.519251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.519251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.572762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] MFA Rules not processed for user `99778ee4e714479abe49af1bc590aeb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:07.574703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:07.642460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1122/2212] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:07 2022] PUT /identity/v3/projects/7037772da7744e1c975187e9a7bf035c/users/b1132fb9b25c4c74a813aec6e8bf7c97/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:07.649739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.715363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1091/2213] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:07.726814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.744694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:07.797697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c8f3d-c2d5-4a18-a7bc-e278f6028d6c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.803500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.804540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.804830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.805112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.805867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.808966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.825527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1123/2214] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:07.827019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:07.839286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4d195-2522-4d66-a33c-806ffbc97024 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.841659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec79c3-7123-42b6-a753-34a3a194dc75 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:07.844166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc12cc-0680-44d1-b8c0-7ee616fce6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.844916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc12cc-0680-44d1-b8c0-7ee616fce6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.845361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc12cc-0680-44d1-b8c0-7ee616fce6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.845653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc12cc-0680-44d1-b8c0-7ee616fce6d2 admin admin] PATH_INFO: `/v3/projects/7037772da7744e1c975187e9a7bf035c/users/b1132fb9b25c4c74a813aec6e8bf7c97/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.847712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc12cc-0680-44d1-b8c0-7ee616fce6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7037772da7744e1c975187e9a7bf035c, user_id=b1132fb9b25c4c74a813aec6e8bf7c97, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:07.850493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc12cc-0680-44d1-b8c0-7ee616fce6d2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:07.864112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1124/2215] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:07 2022] PUT /identity/v3/projects/7037772da7744e1c975187e9a7bf035c/users/b1132fb9b25c4c74a813aec6e8bf7c97/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:07.882090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f68e9-b506-4b71-b0be-7285cd44bf1e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.882404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f68e9-b506-4b71-b0be-7285cd44bf1e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.883213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f68e9-b506-4b71-b0be-7285cd44bf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.925572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:07.926214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:07.926769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:07.927058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:07.954680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c62f68e9-b506-4b71-b0be-7285cd44bf1e None None] MFA Rules not processed for user `b1132fb9b25c4c74a813aec6e8bf7c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:07.956819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c62f68e9-b506-4b71-b0be-7285cd44bf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:08.024095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:08.027499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:08.040810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1092/2216] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:08.044424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1125/2217] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:08.064390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:08.069606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:08.089649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bde69e-da8c-41eb-9af3-9d610ee14db6 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:08.160364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:08.175135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:08.176468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:08.176827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:08.177189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:08.193063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:08.195000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:08.210051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1093/2218] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:08.300879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f68e9-b506-4b71-b0be-7285cd44bf1e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:08.305349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:08.307727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:08.389609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f68e9-b506-4b71-b0be-7285cd44bf1e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:08.394757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:08.395435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:08.395724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:08.396036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] PATH_INFO: `/v3/users/35e28a35b1154cffb33e4db5aebfc423` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:08.398166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e28a35b1154cffb33e4db5aebfc423)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:08.400213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:08.403169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1b4d4-8bcf-44e1-9d4f-b22946845e21 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:08.410429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:08.411171 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:08.411500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:08.411785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] PATH_INFO: `/v3/projects/3c152c0bd9bb436e8bfb41cba0a17036` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:08.412973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c152c0bd9bb436e8bfb41cba0a17036)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:08.414929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:08.610620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c152c0bd9bb436e8bfb41cba0a17036', 'request_id': 'req-78fb8048-397f-4e2f-920b-25d8c8e439b0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.610910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e28a35b1154cffb33e4db5aebfc423'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.618511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e28a35b1154cffb33e4db5aebfc423'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.619241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e28a35b1154cffb33e4db5aebfc423'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.619910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e28a35b1154cffb33e4db5aebfc423'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.620355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c152c0bd9bb436e8bfb41cba0a17036', 'request_id': 'req-78fb8048-397f-4e2f-920b-25d8c8e439b0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.633057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a9791784-e413-4b8c-a381-803c458672cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e28a35b1154cffb33e4db5aebfc423'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.713951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c152c0bd9bb436e8bfb41cba0a17036', 'request_id': 'req-78fb8048-397f-4e2f-920b-25d8c8e439b0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.724907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] The token cache is being invalidate because project 3c152c0bd9bb436e8bfb41cba0a17036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:08.725161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.725761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:08.727309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1094/2219] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:08 2022] DELETE /identity/v3/projects/3c152c0bd9bb436e8bfb41cba0a17036 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:08.728160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:08.729515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:08.798787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb8048-397f-4e2f-920b-25d8c8e439b0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:08.807938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:08.808597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:08.808914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:08.809189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] PATH_INFO: `/v3/projects/fd4bc3eee0f94562913239be525b6438` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:08.812448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4bc3eee0f94562913239be525b6438)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:08.815859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:08.825738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1126/2220] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:08 2022] DELETE /identity/v3/users/35e28a35b1154cffb33e4db5aebfc423 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:08.828822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ad57b-3bdf-4c6d-bfcd-19c4c647fd66 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:08.830961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ad57b-3bdf-4c6d-bfcd-19c4c647fd66 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:08.831449 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ad57b-3bdf-4c6d-bfcd-19c4c647fd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:08.861380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f93ad57b-3bdf-4c6d-bfcd-19c4c647fd66 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:08.861380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f93ad57b-3bdf-4c6d-bfcd-19c4c647fd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:08.946777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1127/2221] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:08.946777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68527327-19d9-4845-85ad-183fda34b7ad None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:08.946777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68527327-19d9-4845-85ad-183fda34b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:08.946777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68527327-19d9-4845-85ad-183fda34b7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:08.971973 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-68527327-19d9-4845-85ad-183fda34b7ad None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:08.973590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-68527327-19d9-4845-85ad-183fda34b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:08.995296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4bc3eee0f94562913239be525b6438', 'request_id': 'req-f72ad295-7ace-44f9-9339-ef8538dd027f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:09.005393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4bc3eee0f94562913239be525b6438', 'request_id': 'req-f72ad295-7ace-44f9-9339-ef8538dd027f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:09.062242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1128/2222] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:09.064097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07327007-3358-4735-bd43-f243a73f0eab None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.064386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07327007-3358-4735-bd43-f243a73f0eab None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.064663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07327007-3358-4735-bd43-f243a73f0eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.086567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-07327007-3358-4735-bd43-f243a73f0eab None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:09.087829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-07327007-3358-4735-bd43-f243a73f0eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:09.096325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1129/2223] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:09.097322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07327007-3358-4735-bd43-f243a73f0eab None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.098593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-07327007-3358-4735-bd43-f243a73f0eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:09.100215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4bc3eee0f94562913239be525b6438', 'request_id': 'req-f72ad295-7ace-44f9-9339-ef8538dd027f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:09.119064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] The token cache is being invalidate because project fd4bc3eee0f94562913239be525b6438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:09.119394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:09.120084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:09.125189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1095/2224] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:08 2022] DELETE /identity/v3/projects/fd4bc3eee0f94562913239be525b6438 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:09.125189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.125189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:09.137805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ad295-7ace-44f9-9339-ef8538dd027f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.142869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76e1a1-a70c-4dd4-a294-1f2ce20959c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.143554 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76e1a1-a70c-4dd4-a294-1f2ce20959c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.143838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76e1a1-a70c-4dd4-a294-1f2ce20959c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.144114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76e1a1-a70c-4dd4-a294-1f2ce20959c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.144813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76e1a1-a70c-4dd4-a294-1f2ce20959c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:09.145261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76e1a1-a70c-4dd4-a294-1f2ce20959c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.146890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76e1a1-a70c-4dd4-a294-1f2ce20959c7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.169043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1096/2225] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:09.170805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b9b2a-ee9c-4bac-89ed-716dda1dcd27 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.171225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b9b2a-ee9c-4bac-89ed-716dda1dcd27 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.171516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b9b2a-ee9c-4bac-89ed-716dda1dcd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.172553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07327007-3358-4735-bd43-f243a73f0eab None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.177183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9750d-5da0-461e-a80d-fa061b86653c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.177813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9750d-5da0-461e-a80d-fa061b86653c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.178106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9750d-5da0-461e-a80d-fa061b86653c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.178394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9750d-5da0-461e-a80d-fa061b86653c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.181217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8ec9750d-5da0-461e-a80d-fa061b86653c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:09.194960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9750d-5da0-461e-a80d-fa061b86653c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.200682 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d65b9b2a-ee9c-4bac-89ed-716dda1dcd27 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:09.200873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9750d-5da0-461e-a80d-fa061b86653c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.200975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d65b9b2a-ee9c-4bac-89ed-716dda1dcd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:09.207147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1097/2226] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:09.207920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1130/2227] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:09.208507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b9b2a-ee9c-4bac-89ed-716dda1dcd27 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.216614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54f9f6-78eb-4c33-8b18-4d053bbe6728 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.216917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54f9f6-78eb-4c33-8b18-4d053bbe6728 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.217203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54f9f6-78eb-4c33-8b18-4d053bbe6728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.225271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b9b2a-ee9c-4bac-89ed-716dda1dcd27 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.237700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.238550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.238889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.240901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.240901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.242597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.243625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ba54f9f6-78eb-4c33-8b18-4d053bbe6728 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:09.245240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ba54f9f6-78eb-4c33-8b18-4d053bbe6728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:09.258541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1131/2228] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:09.263296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54f9f6-78eb-4c33-8b18-4d053bbe6728 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.273258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54f9f6-78eb-4c33-8b18-4d053bbe6728 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.278457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.279209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.279519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.280014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.280781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:09.281957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.283816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.295994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1132/2229] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:09.301433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.311691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875f76-dfde-4eb3-8f4a-cbc0bf462e3f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.317128 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.317816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.318293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.318577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.319580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.327346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.361524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1098/2230] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:09.367349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.440444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca074f4-fd52-4061-994e-78cdca4556ce admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.445207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.445834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.446122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.446404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.447079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.449053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.528034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1133/2231] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:23:09.539467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.590385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1099/2232] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:09.608419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.623618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183417e-374d-4bdc-b44f-6cdc7fac16a2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.638823 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.638990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d48be-e07c-4a7a-af71-55852b418e87 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.639609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.639827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.640354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.651304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.651304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.651501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.651501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.651501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.651501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.651868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.651868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.657727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1100/2233] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:09.659979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.673628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83183a97-7e50-4b04-bbcb-b29d1432b60d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.683019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.683731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.684042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.684321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.685033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.686646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.698084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1101/2234] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:09.709450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.722685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da58c6-a498-4b5e-8a65-97f245c75c18 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.729024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.729746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.730246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.730534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] PATH_INFO: `/v3/projects/4f52a818be854bce9ba1c1405bab4a6c/users/c0fc81b95f6d41bf843bf9aadabcda9d/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.756172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f52a818be854bce9ba1c1405bab4a6c, user_id=c0fc81b95f6d41bf843bf9aadabcda9d, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.759971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.786650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1134/2235] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:09.790829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.802506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce43419-7946-4f2b-a322-d00ec879407e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.811039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.812102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.812202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.812684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.813844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.815065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.826404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1135/2236] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:09.830935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.848027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be82ae-af22-4b61-9935-47a28f3b26e3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:09.852664 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4645e-cab2-4856-a8fc-2a573699378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:09.853317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4645e-cab2-4856-a8fc-2a573699378e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.853602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4645e-cab2-4856-a8fc-2a573699378e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.853873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4645e-cab2-4856-a8fc-2a573699378e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.854617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4645e-cab2-4856-a8fc-2a573699378e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:09.859547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4645e-cab2-4856-a8fc-2a573699378e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:09.887365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1102/2237] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:09 2022] PUT /identity/v3/projects/4f52a818be854bce9ba1c1405bab4a6c/users/c0fc81b95f6d41bf843bf9aadabcda9d/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:09.887365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:09.890485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:09.904071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1136/2238] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:09.904071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8630c5-a079-403c-a763-fe1a690397e0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:09.904071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8630c5-a079-403c-a763-fe1a690397e0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:09.909091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8630c5-a079-403c-a763-fe1a690397e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:09.944178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5c8630c5-a079-403c-a763-fe1a690397e0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:09.946345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5c8630c5-a079-403c-a763-fe1a690397e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:10.015724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae9fc0-5fc5-4b04-8277-0e8d53023fe0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:10.022866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:10.023568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:10.024003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:10.024295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] PATH_INFO: `/v3/projects/f0785be18f464028aaab62d4abe7f133` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:10.025750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0785be18f464028aaab62d4abe7f133)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:10.027979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:10.051228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1137/2239] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:10.051228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8630c5-a079-403c-a763-fe1a690397e0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:10.089056 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8630c5-a079-403c-a763-fe1a690397e0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:10.094157 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:10.097575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:10.097913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:10.098299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] PATH_INFO: `/v3/projects/f250f40ea27c48eb89e5fb195211a6d6/users/e5b153d3c22e4461a88e780fda46db98/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:10.136068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f250f40ea27c48eb89e5fb195211a6d6, user_id=e5b153d3c22e4461a88e780fda46db98, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:10.144537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:10.284422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1138/2240] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:10 2022] PUT /identity/v3/projects/f250f40ea27c48eb89e5fb195211a6d6/users/e5b153d3c22e4461a88e780fda46db98/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:10.285325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:10.292220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:10.296980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0785be18f464028aaab62d4abe7f133', 'request_id': 'req-6f788503-33a0-4492-a850-4768bf520b32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.305019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0785be18f464028aaab62d4abe7f133', 'request_id': 'req-6f788503-33a0-4492-a850-4768bf520b32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.369985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ecbc6-5199-4a4f-b586-a9169132ee50 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:10.426965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:10.427634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:10.427921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:10.428194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:10.445972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0785be18f464028aaab62d4abe7f133', 'request_id': 'req-6f788503-33a0-4492-a850-4768bf520b32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.459428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] The token cache is being invalidate because project f0785be18f464028aaab62d4abe7f133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:10.459958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.460706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6f788503-33a0-4492-a850-4768bf520b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.462427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1103/2241] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:09 2022] DELETE /identity/v3/projects/f0785be18f464028aaab62d4abe7f133 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:10.464646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:10.464936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:10.465213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:10.505814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] MFA Rules not processed for user `e5b153d3c22e4461a88e780fda46db98`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:10.509288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:10.524679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:10.527091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:10.573179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1139/2242] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:10.574234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:10.588513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dbae9-efab-444f-8265-768c44aefd32 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:10.594656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:10.595478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:10.595765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:10.596047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] PATH_INFO: `/v3/users/b1132fb9b25c4c74a813aec6e8bf7c97` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:10.598208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1132fb9b25c4c74a813aec6e8bf7c97)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:10.600698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:10.615345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1104/2243] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:10.628224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:10.631220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:10.699918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6d5c3-02f4-4fea-a1a7-98b31ae3794b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:10.704451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:10.705478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:10.705878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:10.706157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:10.720716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:10.724050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:10.724050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:10.743557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1105/2244] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:10.817566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1132fb9b25c4c74a813aec6e8bf7c97'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.833415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1132fb9b25c4c74a813aec6e8bf7c97'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.834366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1132fb9b25c4c74a813aec6e8bf7c97'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.835264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1132fb9b25c4c74a813aec6e8bf7c97'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:10.853989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c2c72b4d-2840-42ee-9547-7b6b00b52a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1132fb9b25c4c74a813aec6e8bf7c97'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:11.028385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1140/2245] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:10 2022] DELETE /identity/v3/users/b1132fb9b25c4c74a813aec6e8bf7c97 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:11.712267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:11.718042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:11.743115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d37568-023b-4e8a-a808-94540845c49c service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:11.909633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:11.912117 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:11.912624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:11.913723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] PATH_INFO: `/v3/projects/7037772da7744e1c975187e9a7bf035c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:11.915265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7037772da7744e1c975187e9a7bf035c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:11.917428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:12.120095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7037772da7744e1c975187e9a7bf035c', 'request_id': 'req-b698e9b8-802b-4102-a982-d8f881cd8bb8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:12.134728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7037772da7744e1c975187e9a7bf035c', 'request_id': 'req-b698e9b8-802b-4102-a982-d8f881cd8bb8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:12.257972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7037772da7744e1c975187e9a7bf035c', 'request_id': 'req-b698e9b8-802b-4102-a982-d8f881cd8bb8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:12.317661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] The token cache is being invalidate because project 7037772da7744e1c975187e9a7bf035c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:12.318695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:12.319817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b698e9b8-802b-4102-a982-d8f881cd8bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:12.322165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1106/2246] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:11 2022] DELETE /identity/v3/projects/7037772da7744e1c975187e9a7bf035c => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:12.328701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27cb0e-da90-4b00-be28-24d3d2375faa None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:12.329315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27cb0e-da90-4b00-be28-24d3d2375faa None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:12.329315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27cb0e-da90-4b00-be28-24d3d2375faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:12.353019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cb27cb0e-da90-4b00-be28-24d3d2375faa None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:12.357851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cb27cb0e-da90-4b00-be28-24d3d2375faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:12.366977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bab11-6ab5-4410-a71a-fc47fa5830e5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:12.367361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bab11-6ab5-4410-a71a-fc47fa5830e5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:12.367708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bab11-6ab5-4410-a71a-fc47fa5830e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:12.406850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0b2bab11-6ab5-4410-a71a-fc47fa5830e5 None None] MFA Rules not processed for user `d2ee2b6dd6f647179d2af999b3e62788`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:12.409914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0b2bab11-6ab5-4410-a71a-fc47fa5830e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:12.451513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1141/2247] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:12.453241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d938-6ae1-4efc-a77a-dda4ae8b259e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:12.453607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d938-6ae1-4efc-a77a-dda4ae8b259e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:12.453966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d938-6ae1-4efc-a77a-dda4ae8b259e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:12.475135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e0b8d938-6ae1-4efc-a77a-dda4ae8b259e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:12.476369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e0b8d938-6ae1-4efc-a77a-dda4ae8b259e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:12.476713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1107/2248] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:12.484643 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1142/2249] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:12.489638 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bab11-6ab5-4410-a71a-fc47fa5830e5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:12.501018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bab11-6ab5-4410-a71a-fc47fa5830e5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:12.505807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:12.506595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:12.507090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:12.507406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:12.508512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:12.508512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:12.510041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:12.523476 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1108/2250] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:23:12.528948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8d938-6ae1-4efc-a77a-dda4ae8b259e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:12.542517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8d938-6ae1-4efc-a77a-dda4ae8b259e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:12.547681 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:12.548928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:12.549214 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:12.549492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:12.550234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:12.552283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:12.693715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:12.701316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1143/2251] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:12 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:12.706828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:12.719007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf2de5-7b19-4bce-b1b6-877208700cae admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:12.803567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:12.804197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:12.804485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:12.804766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:12.805472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:12.807623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:12.961402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1109/2252] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:12 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:12.965602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:12.976275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0ffa-7d8b-482d-800a-8d5656ac13d2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:12.981486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:12.982290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:12.982888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:12.983286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:12.984083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:12.987491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:13.000788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1144/2253] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:13.002257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:13.017173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8d6a-6ac4-4834-a05d-a875aab901ea admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:13.023405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38abe3-a273-4ee0-ac9f-185653dbbd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:13.024793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38abe3-a273-4ee0-ac9f-185653dbbd26 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:13.025247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38abe3-a273-4ee0-ac9f-185653dbbd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:13.025674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38abe3-a273-4ee0-ac9f-185653dbbd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:13.026471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38abe3-a273-4ee0-ac9f-185653dbbd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:13.028346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38abe3-a273-4ee0-ac9f-185653dbbd26 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:13.043242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1110/2254] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:13.045289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:13.057616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a6d3c-0108-4dd6-b0e7-b01bdfd85ac9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:13.062190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:13.062874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:13.063336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:13.064258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] PATH_INFO: `/v3/projects/f3d142975f4541b69f5a11a995838d3f/users/8d51db72fe944e37a82a9d2ea54b8821/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:13.081319 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d142975f4541b69f5a11a995838d3f, user_id=8d51db72fe944e37a82a9d2ea54b8821, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:13.084356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:13.195866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1145/2255] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:13 2022] PUT /identity/v3/projects/f3d142975f4541b69f5a11a995838d3f/users/8d51db72fe944e37a82a9d2ea54b8821/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:13.211859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d15fe7-971d-4279-b259-da191ff887a3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:13.212327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d15fe7-971d-4279-b259-da191ff887a3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:13.212970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d15fe7-971d-4279-b259-da191ff887a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:13.262606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-81d15fe7-971d-4279-b259-da191ff887a3 None None] MFA Rules not processed for user `8d51db72fe944e37a82a9d2ea54b8821`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:13.264246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-81d15fe7-971d-4279-b259-da191ff887a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:13.394043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:13.396181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:13.399315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1111/2256] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:13.406392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d15fe7-971d-4279-b259-da191ff887a3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:13.513159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d15fe7-971d-4279-b259-da191ff887a3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:13.527758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:13.528903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:13.528999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7f23c-5138-4a81-a6cd-223c4af86cdc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:13.529091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:13.529427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:13.530609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:13.540182 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:13.551988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:13.553136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:13.554841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:13.555325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] PATH_INFO: `/v3/users/99778ee4e714479abe49af1bc590aeb1` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:13.560461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99778ee4e714479abe49af1bc590aeb1)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:13.563719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:13.653480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1112/2257] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:13 2022] POST /identity/v3/projects => generated 385 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:13.656224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:13.754754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54ca4-129a-4d31-bdb8-931b1674df1d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:13.766524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:13.768319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:13.768834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:13.769397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:13.784350 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:13.789104 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:13.804800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99778ee4e714479abe49af1bc590aeb1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:13.814816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99778ee4e714479abe49af1bc590aeb1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:13.815804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99778ee4e714479abe49af1bc590aeb1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:13.816308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99778ee4e714479abe49af1bc590aeb1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:13.832823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99382df2-5d45-4221-8a92-5199f6d29717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99778ee4e714479abe49af1bc590aeb1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:13.929075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1146/2258] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:13 2022] DELETE /identity/v3/users/99778ee4e714479abe49af1bc590aeb1 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:13.933342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318691b2-a937-407a-801e-af923acd477a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:13.935727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1113/2259] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:13 2022] POST /identity/v3/users => generated 299 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:23:13.936070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318691b2-a937-407a-801e-af923acd477a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:13.936572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318691b2-a937-407a-801e-af923acd477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:13.940586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:13.949154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:13.982357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115288e3-f929-424b-bc08-2eb09c5c1b9a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:13.985752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-318691b2-a937-407a-801e-af923acd477a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:13.987572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-318691b2-a937-407a-801e-af923acd477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:14.096502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:14.098679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.099151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.099662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.100732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:14.103779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:14.124209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1114/2260] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:14.126972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:14.140987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1147/2261] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:14.140987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318691b2-a937-407a-801e-af923acd477a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:14.145575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-318691b2-a937-407a-801e-af923acd477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:14.153879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65105c5e-3776-41ff-8cae-55c49dd0ea81 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:14.168289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:14.169161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.169697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.170037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] PATH_INFO: `/v3/projects/c6045adef88d40acb726f0d3c180ddaf/users/07c4843cee144b0bb7437b9ad86208da/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.201994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6045adef88d40acb726f0d3c180ddaf, user_id=07c4843cee144b0bb7437b9ad86208da, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:14.206078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:14.227974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318691b2-a937-407a-801e-af923acd477a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:14.231659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:14.232485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.232754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.233039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.243994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:14.245617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:14.257313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1148/2262] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:14.312750 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1115/2263] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:14 2022] PUT /identity/v3/projects/c6045adef88d40acb726f0d3c180ddaf/users/07c4843cee144b0bb7437b9ad86208da/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:14.314728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:14.331656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e39a1-f1db-4c0b-a1f0-71e7e7017530 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:14.419438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4243f82-e3e5-4adf-8f41-752895cb82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:14.420222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4243f82-e3e5-4adf-8f41-752895cb82dd admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.420529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4243f82-e3e5-4adf-8f41-752895cb82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.420934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4243f82-e3e5-4adf-8f41-752895cb82dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.421879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4243f82-e3e5-4adf-8f41-752895cb82dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:14.435362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4243f82-e3e5-4adf-8f41-752895cb82dd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:14.436489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1149/2264] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:14.440812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:14.463295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a9b88-0fda-4897-a3a7-bf7ce56f81e7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:14.468380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:14.469075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.469355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.469744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] PATH_INFO: `/v3/projects/c6045adef88d40acb726f0d3c180ddaf/users/07c4843cee144b0bb7437b9ad86208da/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.471874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6045adef88d40acb726f0d3c180ddaf, user_id=07c4843cee144b0bb7437b9ad86208da, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:14.474690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:14.487908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1116/2265] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:14 2022] PUT /identity/v3/projects/c6045adef88d40acb726f0d3c180ddaf/users/07c4843cee144b0bb7437b9ad86208da/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:14.498522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.498821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.499109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.552393 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] MFA Rules not processed for user `07c4843cee144b0bb7437b9ad86208da`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:14.553735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:14.641571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:14.649287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:14.693452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1150/2266] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:14.702436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:14.706914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:14.783513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6234a-8de0-4987-b4b6-cd9b5641b707 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:14.793813 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:14.794801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.795102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.795560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] PATH_INFO: `/v3/users/e5b153d3c22e4461a88e780fda46db98` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.797855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b153d3c22e4461a88e780fda46db98)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:14.799464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:14.923397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3f99-52a0-45f4-a166-976f497a8693 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:14.931516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8165759-a199-4f94-9e2a-fddb4dea92e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:14.932319 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8165759-a199-4f94-9e2a-fddb4dea92e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:14.932739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8165759-a199-4f94-9e2a-fddb4dea92e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:14.933132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8165759-a199-4f94-9e2a-fddb4dea92e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:14.935818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c8165759-a199-4f94-9e2a-fddb4dea92e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:15.053141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8165759-a199-4f94-9e2a-fddb4dea92e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:15.054712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8165759-a199-4f94-9e2a-fddb4dea92e0 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:15.079714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1151/2267] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:15.089326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b153d3c22e4461a88e780fda46db98'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:15.091341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b153d3c22e4461a88e780fda46db98'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:15.091462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:15.105385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b153d3c22e4461a88e780fda46db98'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:15.177114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b153d3c22e4461a88e780fda46db98'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:15.199998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b153d3c22e4461a88e780fda46db98'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:15.223775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:15.224769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:15.225154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:15.228731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1117/2268] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:14 2022] DELETE /identity/v3/users/e5b153d3c22e4461a88e780fda46db98 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:15.283415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:15.288155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:15.395967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1152/2269] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:15.433616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:15.435937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:15.631166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59197481-9aa9-4906-8b90-a51cf9e05465 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:15.646939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde570b-7cd2-4322-acfb-d9a1a2a5c76d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:15.648069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde570b-7cd2-4322-acfb-d9a1a2a5c76d service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:15.648567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde570b-7cd2-4322-acfb-d9a1a2a5c76d service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:15.648851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde570b-7cd2-4322-acfb-d9a1a2a5c76d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:15.660771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde570b-7cd2-4322-acfb-d9a1a2a5c76d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:15.662356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde570b-7cd2-4322-acfb-d9a1a2a5c76d service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:15.702912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1118/2270] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:15.787941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:15.805834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:15.837511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1009d8-fb05-4038-8583-557a753d9364 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:15.851977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:15.853812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:15.855082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:15.856045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] PATH_INFO: `/v3/projects/871a7bb725ba408eb421b2c94d6ccad7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:15.859642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] RBAC: Authorizing `identity:delete_project(project_id=871a7bb725ba408eb421b2c94d6ccad7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:15.865791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:16.063355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871a7bb725ba408eb421b2c94d6ccad7', 'request_id': 'req-6c28c8da-c247-4f76-a98c-98ebd886344a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:16.247436 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871a7bb725ba408eb421b2c94d6ccad7', 'request_id': 'req-6c28c8da-c247-4f76-a98c-98ebd886344a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:16.299977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871a7bb725ba408eb421b2c94d6ccad7', 'request_id': 'req-6c28c8da-c247-4f76-a98c-98ebd886344a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:16.329367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] The token cache is being invalidate because project 871a7bb725ba408eb421b2c94d6ccad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:16.329676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:16.331127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c28c8da-c247-4f76-a98c-98ebd886344a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:16.333228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1153/2271] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:15 2022] DELETE /identity/v3/projects/871a7bb725ba408eb421b2c94d6ccad7 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:16.338456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ebf8a-e4b9-4818-9688-4d38d892e30f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:16.338967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ebf8a-e4b9-4818-9688-4d38d892e30f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:16.338967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ebf8a-e4b9-4818-9688-4d38d892e30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:16.396911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cc7ebf8a-e4b9-4818-9688-4d38d892e30f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:16.403529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cc7ebf8a-e4b9-4818-9688-4d38d892e30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:16.539675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bd531-330a-476d-9b54-1534829b791d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:16.540077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bd531-330a-476d-9b54-1534829b791d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:16.540654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bd531-330a-476d-9b54-1534829b791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:16.550332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1119/2272] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:16.591217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-468bd531-330a-476d-9b54-1534829b791d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:16.592829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-468bd531-330a-476d-9b54-1534829b791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:16.624224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1154/2273] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:16.630367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ebf8a-e4b9-4818-9688-4d38d892e30f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:16.644867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ebf8a-e4b9-4818-9688-4d38d892e30f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:16.650345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:16.651512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:16.652182 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:16.653815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:16.654649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:16.655127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:16.655668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:16.659160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:16.680935 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1120/2274] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:16.684818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bd531-330a-476d-9b54-1534829b791d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:16.694959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bd531-330a-476d-9b54-1534829b791d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:16.718585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:16.718944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:16.719025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:16.719292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:16.721195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:16.724752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:16.746162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:16.747940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:16.762860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76b60-1d51-42d8-98f6-f26f4d3bc68c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:16.776981 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:16.777672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:16.777983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:16.778270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] PATH_INFO: `/v3/projects/f250f40ea27c48eb89e5fb195211a6d6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:16.779793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f250f40ea27c48eb89e5fb195211a6d6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:16.781866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:16.841228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1155/2275] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:16 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:16.848011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.007746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5e420-c894-44b1-b94c-fdb1d4648ef0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.027555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f250f40ea27c48eb89e5fb195211a6d6', 'request_id': 'req-ecca22a9-0156-4455-b240-defd0b090da4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:17.044628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f250f40ea27c48eb89e5fb195211a6d6', 'request_id': 'req-ecca22a9-0156-4455-b240-defd0b090da4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:17.143719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f250f40ea27c48eb89e5fb195211a6d6', 'request_id': 'req-ecca22a9-0156-4455-b240-defd0b090da4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:17.160435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] The token cache is being invalidate because project f250f40ea27c48eb89e5fb195211a6d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:17.160435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:17.161681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ecca22a9-0156-4455-b240-defd0b090da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:17.166260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1121/2276] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:16 2022] DELETE /identity/v3/projects/f250f40ea27c48eb89e5fb195211a6d6 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:17.171319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bfb08-9d02-4ecf-afc8-82fe0e688b48 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.171472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bfb08-9d02-4ecf-afc8-82fe0e688b48 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.171766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bfb08-9d02-4ecf-afc8-82fe0e688b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.201313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.201958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.202241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.202521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.203219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.208282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.209199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e41bfb08-9d02-4ecf-afc8-82fe0e688b48 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:17.210661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e41bfb08-9d02-4ecf-afc8-82fe0e688b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:17.227254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1122/2277] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:17.229195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d2460-01ac-4e34-80cb-e47d727c8ca0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.229494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d2460-01ac-4e34-80cb-e47d727c8ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.229964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d2460-01ac-4e34-80cb-e47d727c8ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.290071 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2f4d2460-01ac-4e34-80cb-e47d727c8ca0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:17.297576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2f4d2460-01ac-4e34-80cb-e47d727c8ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:17.339111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1156/2278] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:16 2022] POST /identity/v3/users => generated 283 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:17.343291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.347393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1123/2279] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:17.347894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:17.355868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d2460-01ac-4e34-80cb-e47d727c8ca0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.370178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad070873-5d66-4117-b7f8-32c785a44fba admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.389877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d2460-01ac-4e34-80cb-e47d727c8ca0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.390275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.391165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.391655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.391936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.392767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.394553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.397384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.398785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.399473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.401812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.401812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:17.401812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.403124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.419798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1157/2280] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:17.421522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.423429 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1124/2281] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:17.428469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.444268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542f50-d00b-451e-8057-bbee0118e6fb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.444452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9929a-b31e-4272-97bf-b3f4d998e798 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.452743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.453786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.454333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.454835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.455941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.459165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.459320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.459320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.459320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.459320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.465168 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.465473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.489924 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1158/2282] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:17.493718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.517232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5dea0-45d1-4e3a-b789-2be185e57c42 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.535529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bc6dc-1f3d-48ee-9340-84df46256bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.535529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bc6dc-1f3d-48ee-9340-84df46256bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.535529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bc6dc-1f3d-48ee-9340-84df46256bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.535529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bc6dc-1f3d-48ee-9340-84df46256bbd admin admin] PATH_INFO: `/v3/projects/74e053ef85664c65928a62b9c54a9f9e/users/5500c79a37db423e8812128a1d0314f8/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.574642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bc6dc-1f3d-48ee-9340-84df46256bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e053ef85664c65928a62b9c54a9f9e, user_id=5500c79a37db423e8812128a1d0314f8, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.578036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bc6dc-1f3d-48ee-9340-84df46256bbd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.582573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1125/2283] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:17 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:17.586739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.682124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1159/2284] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:17 2022] PUT /identity/v3/projects/74e053ef85664c65928a62b9c54a9f9e/users/5500c79a37db423e8812128a1d0314f8/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:17.689362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aedb4-6f61-4bb8-b758-6130d15f97d3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.696796 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.697089 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.697369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.738331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] MFA Rules not processed for user `5500c79a37db423e8812128a1d0314f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:17.745594 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:17.816859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.817500 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.817784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.818065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.818742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.820749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.845798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1160/2285] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:17.859533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.861610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:17.944966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1126/2286] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:17 2022] POST /identity/v3/users => generated 296 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:17.949483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:17.968936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca07d5a-7530-4c9b-9e49-b1d3d610ae54 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.981032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:17.981691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:17.983170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:17.983170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:17.983170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:17.984618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:17.994188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ab8da-846a-409c-a46e-b710068b0797 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:17.996693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1127/2287] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:18.000648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:18.001057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:18.002052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:18.002516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:18.003015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:18.014549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:18.016077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:18.041431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1161/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:18.060317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eafe91-8cbd-43db-ab2c-935417ea6532 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:18.076404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429edc8-ee44-4ba1-b7d7-12a53a3c8617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:18.077087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429edc8-ee44-4ba1-b7d7-12a53a3c8617 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:18.077373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429edc8-ee44-4ba1-b7d7-12a53a3c8617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:18.077700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429edc8-ee44-4ba1-b7d7-12a53a3c8617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:18.082542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429edc8-ee44-4ba1-b7d7-12a53a3c8617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:18.085908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429edc8-ee44-4ba1-b7d7-12a53a3c8617 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:18.109298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1128/2289] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:18.116311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:18.130187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087916-be30-4f22-91e0-ddc91dff3ed4 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:18.139063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:18.143456 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:18.146187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:18.146575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:18.146932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] PATH_INFO: `/v3/projects/60e824e8701346dcab67222bc77f31ca/users/1f5e0995601b4f26aa4a8da058b65e8b/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:18.175286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e824e8701346dcab67222bc77f31ca, user_id=1f5e0995601b4f26aa4a8da058b65e8b, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:18.180101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:18.322514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1162/2290] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:18 2022] PUT /identity/v3/projects/60e824e8701346dcab67222bc77f31ca/users/1f5e0995601b4f26aa4a8da058b65e8b/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:18.334521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11194fc1-4b2b-4814-81f1-59aae9072f25 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:18.334811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11194fc1-4b2b-4814-81f1-59aae9072f25 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:18.335075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11194fc1-4b2b-4814-81f1-59aae9072f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:18.433997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-11194fc1-4b2b-4814-81f1-59aae9072f25 None None] MFA Rules not processed for user `1f5e0995601b4f26aa4a8da058b65e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:18.435308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-11194fc1-4b2b-4814-81f1-59aae9072f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:18.628409 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1129/2291] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:18.644267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:18.646389 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:18.817199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3679c-db86-4200-94a1-88ac5d6210f6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:18.820911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:18.821468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:18.821784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:18.822049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:18.832197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:18.833756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:18.844085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1163/2292] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:19.716593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:20.456721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4398a-79d4-49c7-a640-e196baf22ab6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:20.457012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4398a-79d4-49c7-a640-e196baf22ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:20.457296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4398a-79d4-49c7-a640-e196baf22ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:20.490956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-bee4398a-79d4-49c7-a640-e196baf22ab6 None None] MFA Rules not processed for user `c0fc81b95f6d41bf843bf9aadabcda9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:20.492354 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bee4398a-79d4-49c7-a640-e196baf22ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:20.573604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1130/2293] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:20.578276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:20.581062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:20.675899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8cdd-f5ad-4242-81d5-20dba1daf961 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:20.680996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:20.681826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:20.682113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:20.682391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:20.683258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:20.685247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:20.874656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1164/2294] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:20 2022] POST /identity/v3/projects => generated 385 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:23:20.875834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4398a-79d4-49c7-a640-e196baf22ab6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:20.919174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4398a-79d4-49c7-a640-e196baf22ab6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:21.010870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:21.011994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:21.012456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:21.012893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:21.013790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:21.025726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:21.145207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:21.146642 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1131/2295] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:20 2022] POST /identity/v3/users => generated 299 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:21.162336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b75a7-c2b7-44af-b25a-1cef83f58cfb admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:21.169944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:21.170662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:21.170959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:21.171277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:21.171993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:21.174138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:21.195297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1165/2296] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:21.200699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:21.220814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6f7c8-1141-4a49-93b5-0a548617339a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:21.226988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:21.228250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:21.228637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:21.229025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] PATH_INFO: `/v3/projects/ac73fae8fccc4dbea1233c0bc775b436/users/549cccc5f6c84daf85d3af1b8bdeb3dd/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:21.295972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac73fae8fccc4dbea1233c0bc775b436, user_id=549cccc5f6c84daf85d3af1b8bdeb3dd, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:21.300954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:21.518644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1132/2297] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:21 2022] PUT /identity/v3/projects/ac73fae8fccc4dbea1233c0bc775b436/users/549cccc5f6c84daf85d3af1b8bdeb3dd/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:21.520121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:21.538605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714bf68-6aef-4083-ab28-575711b00155 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:21.542663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:21.653276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:21.668744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:21.670780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:21.671451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:21.671964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:21.673291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:21.677993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:21.698015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1166/2298] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:21.700806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:21.714461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a18d4d-a631-4205-a85d-348a027e7276 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:21.721632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ac279-f80f-4247-911b-565893825d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:21.722437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ac279-f80f-4247-911b-565893825d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:21.722955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ac279-f80f-4247-911b-565893825d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:21.722955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ac279-f80f-4247-911b-565893825d61 admin admin] PATH_INFO: `/v3/projects/ac73fae8fccc4dbea1233c0bc775b436/users/549cccc5f6c84daf85d3af1b8bdeb3dd/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:21.724039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b63c3d-5cf2-4226-b37a-5e36cd7f2291 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:21.726420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ac279-f80f-4247-911b-565893825d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac73fae8fccc4dbea1233c0bc775b436, user_id=549cccc5f6c84daf85d3af1b8bdeb3dd, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:21.728681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:21.729732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:21.735022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ac279-f80f-4247-911b-565893825d61 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:21.735160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:21.735160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:21.735160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:21.744118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1167/2299] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:21 2022] PUT /identity/v3/projects/ac73fae8fccc4dbea1233c0bc775b436/users/549cccc5f6c84daf85d3af1b8bdeb3dd/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:21.816423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:21.817843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:21.828575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1133/2300] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:22.195288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:25.173869 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:25.805863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:26.539756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0211d-ae8c-4653-b834-2badc51f9d13 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:26.540903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0211d-ae8c-4653-b834-2badc51f9d13 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:26.541801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0211d-ae8c-4653-b834-2badc51f9d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:26.588171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2eb0211d-ae8c-4653-b834-2badc51f9d13 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:26.589614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2eb0211d-ae8c-4653-b834-2badc51f9d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:26.717834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1168/2301] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:23:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:26.788063 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:26.789643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:26.862395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32273c5f-03fb-4751-9179-2feca0d9037c service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:26.944257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:26.947270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:26.947270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:26.947270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:26.955020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:26.956747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:26.966347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1134/2302] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:23:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:23:27.270741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:27.738050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:29.900112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:30.200270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:30.705983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072eb0d-95a2-446e-aac2-4b532c13d148 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:30.705983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072eb0d-95a2-446e-aac2-4b532c13d148 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:30.705983 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072eb0d-95a2-446e-aac2-4b532c13d148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:30.757949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0072eb0d-95a2-446e-aac2-4b532c13d148 None None] MFA Rules not processed for user `549cccc5f6c84daf85d3af1b8bdeb3dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:30.760698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:30.761885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0072eb0d-95a2-446e-aac2-4b532c13d148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:30.910536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1169/2303] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:30.930092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:30.931725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:31.039840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daca14-d96f-4512-8ced-6b3c58d5cbb3 service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:31.044165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:31.045259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:31.046622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:31.047163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:31.103300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:31.139073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:31.140444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:31.155948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1135/2304] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:31.619039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:31.619727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:31.620013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:31.650965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:31.654592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:31.667402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1170/2305] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:23:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:31.678885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:31.694849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b462c7-122a-48eb-8c74-39c31f98b683 service glance] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:31.701460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf7f4d-ea8e-4387-b1d5-1043a2ab04c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:31.702533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7f4d-ea8e-4387-b1d5-1043a2ab04c5 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:31.702829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7f4d-ea8e-4387-b1d5-1043a2ab04c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:31.703112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7f4d-ea8e-4387-b1d5-1043a2ab04c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:31.713832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf7f4d-ea8e-4387-b1d5-1043a2ab04c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:31.715780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf7f4d-ea8e-4387-b1d5-1043a2ab04c5 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:31.743679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1136/2306] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:23:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:23:31.773454 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:32.777470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:34.745460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:35.835607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:38.732239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:38.863891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:39.606753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:39.642693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:40.120934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:40.156516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:40.509566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:40.677763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:40.865112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:42.189087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:43.144605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:43.241477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:43.269308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:43.349423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:43.514908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:46.089580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:46.090566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:46.105675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:46.168829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163124e4-3cda-479f-9c30-1ec49dc58e04 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:46.173954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:46.174638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:46.174929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:46.176327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] PATH_INFO: `/v3/users/8d51db72fe944e37a82a9d2ea54b8821` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:46.177974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d51db72fe944e37a82a9d2ea54b8821)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:46.179936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:47.153505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d51db72fe944e37a82a9d2ea54b8821'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.163235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d51db72fe944e37a82a9d2ea54b8821'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.164220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d51db72fe944e37a82a9d2ea54b8821'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.164943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d51db72fe944e37a82a9d2ea54b8821'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.197526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d51db72fe944e37a82a9d2ea54b8821'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.358249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1171/2307] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:46 2022] DELETE /identity/v3/users/8d51db72fe944e37a82a9d2ea54b8821 => generated 0 bytes in 1270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:47.362919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:47.363799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:47.364043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:47.420139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:47.421597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:47.505630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1137/2308] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:47.507709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:47.512466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:47.626637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfc955-2248-43a5-8116-1dbae4ce5c77 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:47.630781 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c64ee-6a28-420d-bb2d-da6f3bb8227f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:47.648884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c64ee-6a28-420d-bb2d-da6f3bb8227f service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:47.648884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c64ee-6a28-420d-bb2d-da6f3bb8227f service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:47.648884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c64ee-6a28-420d-bb2d-da6f3bb8227f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:47.648884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c64ee-6a28-420d-bb2d-da6f3bb8227f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:47.648884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c64ee-6a28-420d-bb2d-da6f3bb8227f service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:47.658338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1172/2309] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:48.173678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:48.176294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:48.201345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170bc70-1196-4913-94e0-07b7304c44cb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:48.208834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:48.209778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:48.210364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:48.210744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] PATH_INFO: `/v3/projects/f3d142975f4541b69f5a11a995838d3f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:48.212390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d142975f4541b69f5a11a995838d3f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:48.214624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:48.353794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:48.368365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d142975f4541b69f5a11a995838d3f', 'request_id': 'req-60ed2fb9-1796-4bad-8563-f6d58481223e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.375711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d142975f4541b69f5a11a995838d3f', 'request_id': 'req-60ed2fb9-1796-4bad-8563-f6d58481223e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.403253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb2e4f-2819-4c28-9959-de03b616aa95 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:48.403835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb2e4f-2819-4c28-9959-de03b616aa95 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:48.403835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb2e4f-2819-4c28-9959-de03b616aa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:48.423580 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-12cb2e4f-2819-4c28-9959-de03b616aa95 None None] MFA Rules not processed for user `c0fc81b95f6d41bf843bf9aadabcda9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:48.424869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-12cb2e4f-2819-4c28-9959-de03b616aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:48.456408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:48.461133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d142975f4541b69f5a11a995838d3f', 'request_id': 'req-60ed2fb9-1796-4bad-8563-f6d58481223e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.469571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] The token cache is being invalidate because project f3d142975f4541b69f5a11a995838d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:48.470392 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.471066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.475799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1138/2310] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:48 2022] DELETE /identity/v3/projects/f3d142975f4541b69f5a11a995838d3f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:48.476887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:48.478356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:48.487612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1173/2311] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:48.576671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2fb9-1796-4bad-8563-f6d58481223e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:48.582323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:48.582581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:48.582681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:48.588672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] PATH_INFO: `/v3/users/07c4843cee144b0bb7437b9ad86208da` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:48.590033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c4843cee144b0bb7437b9ad86208da)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:48.592046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:48.841510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c4843cee144b0bb7437b9ad86208da'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.842467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c4843cee144b0bb7437b9ad86208da'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.852336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c4843cee144b0bb7437b9ad86208da'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.983944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c4843cee144b0bb7437b9ad86208da'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:49.002909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f209eef5-ea2e-428c-b10b-7e3fa1b226f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c4843cee144b0bb7437b9ad86208da'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:49.010589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1139/2312] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:48 2022] DELETE /identity/v3/users/07c4843cee144b0bb7437b9ad86208da => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:50.042659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb2e4f-2819-4c28-9959-de03b616aa95 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.055362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb2e4f-2819-4c28-9959-de03b616aa95 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.120859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.121508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.121792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.122066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] PATH_INFO: `/v3/projects/c6045adef88d40acb726f0d3c180ddaf` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.123247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6045adef88d40acb726f0d3c180ddaf)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.125175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.239539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6045adef88d40acb726f0d3c180ddaf', 'request_id': 'req-bde1c65b-8d73-4f67-acda-5aaeda04533e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:50.365782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6045adef88d40acb726f0d3c180ddaf', 'request_id': 'req-bde1c65b-8d73-4f67-acda-5aaeda04533e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:50.372313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6045adef88d40acb726f0d3c180ddaf', 'request_id': 'req-bde1c65b-8d73-4f67-acda-5aaeda04533e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:50.384599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] The token cache is being invalidate because project c6045adef88d40acb726f0d3c180ddaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:50.384896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:50.385551 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bde1c65b-8d73-4f67-acda-5aaeda04533e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:50.387401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1174/2313] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:50 2022] DELETE /identity/v3/projects/c6045adef88d40acb726f0d3c180ddaf => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:50.393557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d1e3e-c870-4a4a-a787-7e99298db1df None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.394128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d1e3e-c870-4a4a-a787-7e99298db1df None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.394128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d1e3e-c870-4a4a-a787-7e99298db1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.436066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-af7d1e3e-c870-4a4a-a787-7e99298db1df None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:50.436477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-af7d1e3e-c870-4a4a-a787-7e99298db1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:50.480209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:50.505849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1140/2314] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:50.514197 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd982b8-dd53-43bb-abcb-384aea151044 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.514482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd982b8-dd53-43bb-abcb-384aea151044 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.514758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd982b8-dd53-43bb-abcb-384aea151044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.537798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cbd982b8-dd53-43bb-abcb-384aea151044 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:50.538968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cbd982b8-dd53-43bb-abcb-384aea151044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:50.547248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1175/2315] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:50.551871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d1e3e-c870-4a4a-a787-7e99298db1df None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.561765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d1e3e-c870-4a4a-a787-7e99298db1df None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.566963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.567621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.567909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.568186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.568894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:50.569200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.571092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.587628 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1141/2316] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:50.588797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd982b8-dd53-43bb-abcb-384aea151044 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.599086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd982b8-dd53-43bb-abcb-384aea151044 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.604108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.616043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.628381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.628381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.628381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.628381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.774886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1176/2317] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:50 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:23:50.778798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.788984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f611f-5bbd-46d6-ab81-d1064c791396 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.846467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.847260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.847558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.847836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.848534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.850502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.961473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1142/2318] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:50 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:50.966081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.978357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d359de6-87fc-4c05-8738-51e2311aaa4e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.983136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.983866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.986247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.986247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.986247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.986800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.998145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1177/2319] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:51.002668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.013188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5643-ca37-4549-be58-a4c13a6af1ef admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.018604 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba38c3-dc84-400c-b547-9f58b9d05355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.019261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba38c3-dc84-400c-b547-9f58b9d05355 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.019674 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba38c3-dc84-400c-b547-9f58b9d05355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.019964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba38c3-dc84-400c-b547-9f58b9d05355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.020696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba38c3-dc84-400c-b547-9f58b9d05355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.022366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba38c3-dc84-400c-b547-9f58b9d05355 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.033204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1143/2320] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:51.037633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.049161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6abe8-1e04-4bec-828e-c94ad710cf28 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.053044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.053699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.053980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.054258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] PATH_INFO: `/v3/projects/0beb8b030c904f1690fa173f4e1a2502/users/dbc0250f955e4a1bb0a1bdb0db04b991/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.073716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0beb8b030c904f1690fa173f4e1a2502, user_id=dbc0250f955e4a1bb0a1bdb0db04b991, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.078365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.149319 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1178/2321] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:51 2022] PUT /identity/v3/projects/0beb8b030c904f1690fa173f4e1a2502/users/dbc0250f955e4a1bb0a1bdb0db04b991/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:51.160700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9677e0-1135-4486-81b4-6a4c1ce9087b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.160998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9677e0-1135-4486-81b4-6a4c1ce9087b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.161283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9677e0-1135-4486-81b4-6a4c1ce9087b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.190548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fd9677e0-1135-4486-81b4-6a4c1ce9087b None None] MFA Rules not processed for user `dbc0250f955e4a1bb0a1bdb0db04b991`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:51.191829 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fd9677e0-1135-4486-81b4-6a4c1ce9087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:51.256158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1144/2322] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:51.267663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.269335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:51.279201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:51.326729 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d6ffc-cc28-41ef-b219-9d500e135879 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.330352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd233c-0e17-48fa-9482-fd51042ba725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.330965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd233c-0e17-48fa-9482-fd51042ba725 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.331299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd233c-0e17-48fa-9482-fd51042ba725 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.350524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd233c-0e17-48fa-9482-fd51042ba725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.350524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd233c-0e17-48fa-9482-fd51042ba725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.350524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd233c-0e17-48fa-9482-fd51042ba725 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.350524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1179/2323] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:51.372437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9677e0-1135-4486-81b4-6a4c1ce9087b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.413301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:51.437263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9677e0-1135-4486-81b4-6a4c1ce9087b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.442778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.443458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.443747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.444025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] PATH_INFO: `/v3/users/dbc0250f955e4a1bb0a1bdb0db04b991` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.445257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc0250f955e4a1bb0a1bdb0db04b991)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.447210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.626639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc0250f955e4a1bb0a1bdb0db04b991'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.627320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc0250f955e4a1bb0a1bdb0db04b991'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.634214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc0250f955e4a1bb0a1bdb0db04b991'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.720892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc0250f955e4a1bb0a1bdb0db04b991'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.734193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc0250f955e4a1bb0a1bdb0db04b991'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.741888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1145/2324] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:51 2022] DELETE /identity/v3/users/dbc0250f955e4a1bb0a1bdb0db04b991 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:51.746790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.747084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.751758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.768084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:51.769309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:51.848235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1180/2325] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:51.857951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.859457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:51.948650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a246775-4305-4740-9d52-5f19f8257941 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.952298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fd09f-44fc-44b7-9800-5a6dbc0832ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.952922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fd09f-44fc-44b7-9800-5a6dbc0832ca service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.953208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fd09f-44fc-44b7-9800-5a6dbc0832ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.953489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fd09f-44fc-44b7-9800-5a6dbc0832ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.963706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fd09f-44fc-44b7-9800-5a6dbc0832ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.965142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fd09f-44fc-44b7-9800-5a6dbc0832ca service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.975600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1146/2326] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:52.648226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:52.650039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:52.668220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f19ae-0996-4e1a-9b73-93d9db39f218 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:52.674803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:52.675930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.676412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.676966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] PATH_INFO: `/v3/projects/0beb8b030c904f1690fa173f4e1a2502` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.678994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0beb8b030c904f1690fa173f4e1a2502)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:52.682788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:52.777797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0beb8b030c904f1690fa173f4e1a2502', 'request_id': 'req-905bed2a-e680-4089-9e85-b099c337bb34'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.863109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0beb8b030c904f1690fa173f4e1a2502', 'request_id': 'req-905bed2a-e680-4089-9e85-b099c337bb34'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.869439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0beb8b030c904f1690fa173f4e1a2502', 'request_id': 'req-905bed2a-e680-4089-9e85-b099c337bb34'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.881290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] The token cache is being invalidate because project 0beb8b030c904f1690fa173f4e1a2502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:52.881584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.882236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-905bed2a-e680-4089-9e85-b099c337bb34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.884059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1181/2327] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:52 2022] DELETE /identity/v3/projects/0beb8b030c904f1690fa173f4e1a2502 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:52.888784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f96c-5171-4dc0-922e-aa78cbf3142b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.889286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f96c-5171-4dc0-922e-aa78cbf3142b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.889391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f96c-5171-4dc0-922e-aa78cbf3142b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.909132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5463f96c-5171-4dc0-922e-aa78cbf3142b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:52.910381 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5463f96c-5171-4dc0-922e-aa78cbf3142b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:52.980231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:52.990624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1147/2328] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:53.003786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273deea3-6794-4f08-9df4-02cb9f4f83a6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.004268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273deea3-6794-4f08-9df4-02cb9f4f83a6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.004669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273deea3-6794-4f08-9df4-02cb9f4f83a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.026909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-273deea3-6794-4f08-9df4-02cb9f4f83a6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:53.029286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-273deea3-6794-4f08-9df4-02cb9f4f83a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:53.038006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1182/2329] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:53.042401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f96c-5171-4dc0-922e-aa78cbf3142b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.053806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f96c-5171-4dc0-922e-aa78cbf3142b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.058741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.059596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.069129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.069129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.069129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:53.069129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.069129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.075757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1148/2330] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:53.080151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273deea3-6794-4f08-9df4-02cb9f4f83a6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.090066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273deea3-6794-4f08-9df4-02cb9f4f83a6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.095494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.096178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.096459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.096732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.097405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.099442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.233840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1183/2331] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:53 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:53.237201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.249070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c792d-e0b6-4498-b8ba-46e60473da00 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.311804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.312435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.312808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.313090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.313768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.315718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.528091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1149/2332] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:53 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:53.532394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.540822 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:53.545012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced6cb9-c78a-4561-9635-edc39c0a9712 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.549651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.550274 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.550560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.550864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.551627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.553228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.564583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1184/2333] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:53.573129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.584388 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04512db6-4517-4abe-8884-ecbdb2e4a897 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.589263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ec0de-ff22-4ad2-bf82-dc12e604e1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.589892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ec0de-ff22-4ad2-bf82-dc12e604e1cf admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.590183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ec0de-ff22-4ad2-bf82-dc12e604e1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.590550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ec0de-ff22-4ad2-bf82-dc12e604e1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.591597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ec0de-ff22-4ad2-bf82-dc12e604e1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.592905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ec0de-ff22-4ad2-bf82-dc12e604e1cf admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.605413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1150/2334] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:53.609640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.620633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e276ef9-bc99-44f1-88a0-226f97dadb92 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.627630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.627630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.627630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.627630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] PATH_INFO: `/v3/projects/b63a11e1f15d4536ba7d875abc5bb7b4/users/37bf163d0df648b6a39f29c0697537f8/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.642679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63a11e1f15d4536ba7d875abc5bb7b4, user_id=37bf163d0df648b6a39f29c0697537f8, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.645597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.742336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1185/2335] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:53 2022] PUT /identity/v3/projects/b63a11e1f15d4536ba7d875abc5bb7b4/users/37bf163d0df648b6a39f29c0697537f8/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:53.756639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.756932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.757213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.794044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] MFA Rules not processed for user `37bf163d0df648b6a39f29c0697537f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:53.795355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:53.860662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1151/2336] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:53.872531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.874278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:53.934606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44da5c3-6d42-4913-bb8a-7080ff3fd0ee admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.939725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef174687-2afa-4812-8a0d-7573a927abf7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.940432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174687-2afa-4812-8a0d-7573a927abf7 service glance] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.940808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174687-2afa-4812-8a0d-7573a927abf7 service glance] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.941092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174687-2afa-4812-8a0d-7573a927abf7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.950080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef174687-2afa-4812-8a0d-7573a927abf7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.951601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef174687-2afa-4812-8a0d-7573a927abf7 service glance] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.961337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1186/2337] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:54.717356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:54.718885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:54.785317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:54.803560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dd8b8-2a3d-4c8c-978b-915c6007d53a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:54.808163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:54.808840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:54.809124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:54.809401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] PATH_INFO: `/v3/users/1f5e0995601b4f26aa4a8da058b65e8b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:54.810658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f5e0995601b4f26aa4a8da058b65e8b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:54.812634 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:55.025103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f5e0995601b4f26aa4a8da058b65e8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:55.027897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f5e0995601b4f26aa4a8da058b65e8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:55.037047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f5e0995601b4f26aa4a8da058b65e8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:55.118792 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f5e0995601b4f26aa4a8da058b65e8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:55.131439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f5e0995601b4f26aa4a8da058b65e8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:55.148659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1152/2338] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:54 2022] DELETE /identity/v3/users/1f5e0995601b4f26aa4a8da058b65e8b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:55.153515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba1419-d746-4fbd-bf93-6618bdd59632 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:55.153515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba1419-d746-4fbd-bf93-6618bdd59632 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:55.154921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba1419-d746-4fbd-bf93-6618bdd59632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:55.177520 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a7ba1419-d746-4fbd-bf93-6618bdd59632 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:55.178815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a7ba1419-d746-4fbd-bf93-6618bdd59632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:55.244957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1187/2339] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:55.255338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:55.257000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:55.319226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33001-e029-4fdb-b78a-0cc96ca23b30 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:55.323238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:55.324406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:55.324762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:55.325166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:55.346881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:55.348912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:55.365015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1153/2340] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:55.681463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:55.681968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:55.684950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:55.716954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:55.718941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:55.812929 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1188/2341] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:23:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:55.844703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:55.847994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:55.939517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773291f-3a11-4ba4-ac80-1c11572e6a18 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:55.945318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916de542-738e-4f8b-bd67-f1250fbfe1a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:55.946070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916de542-738e-4f8b-bd67-f1250fbfe1a1 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:55.946426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916de542-738e-4f8b-bd67-f1250fbfe1a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:55.946710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916de542-738e-4f8b-bd67-f1250fbfe1a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:55.956098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916de542-738e-4f8b-bd67-f1250fbfe1a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:55.957810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916de542-738e-4f8b-bd67-f1250fbfe1a1 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:55.966508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1154/2342] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:23:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:23:56.139795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:56.141712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:56.199952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6823ad2-b58c-48e1-9054-647a2972cfa5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:56.212909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:56.213588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:56.213878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:56.214162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] PATH_INFO: `/v3/projects/60e824e8701346dcab67222bc77f31ca` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:56.218062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e824e8701346dcab67222bc77f31ca)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:56.220658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:56.253410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:56.493207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e824e8701346dcab67222bc77f31ca', 'request_id': 'req-ee144622-0505-4839-9204-cf1c5f2cc67b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:56.589527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e824e8701346dcab67222bc77f31ca', 'request_id': 'req-ee144622-0505-4839-9204-cf1c5f2cc67b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:56.597395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e824e8701346dcab67222bc77f31ca', 'request_id': 'req-ee144622-0505-4839-9204-cf1c5f2cc67b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:56.627292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] The token cache is being invalidate because project 60e824e8701346dcab67222bc77f31ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:56.627787 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:56.631625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee144622-0505-4839-9204-cf1c5f2cc67b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:56.634600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1189/2343] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:56 2022] DELETE /identity/v3/projects/60e824e8701346dcab67222bc77f31ca => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:56.638751 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f354918-fe67-4b31-944d-e5693d8c24ca None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:56.639868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f354918-fe67-4b31-944d-e5693d8c24ca None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:56.641188 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f354918-fe67-4b31-944d-e5693d8c24ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:56.676657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:56.702963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2f354918-fe67-4b31-944d-e5693d8c24ca None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:56.707720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2f354918-fe67-4b31-944d-e5693d8c24ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:56.827925 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1155/2344] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:56.837209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb2c2-ed9a-4574-a4a1-0fb52ffa0925 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:56.837495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb2c2-ed9a-4574-a4a1-0fb52ffa0925 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:56.837952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb2c2-ed9a-4574-a4a1-0fb52ffa0925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:56.858873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dc1cb2c2-ed9a-4574-a4a1-0fb52ffa0925 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:56.860529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dc1cb2c2-ed9a-4574-a4a1-0fb52ffa0925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:56.873720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1190/2345] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:56.881615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f354918-fe67-4b31-944d-e5693d8c24ca None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:56.906116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc7fc-b46a-46e9-aff5-23500341b18b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:56.907689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc7fc-b46a-46e9-aff5-23500341b18b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:56.909869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc7fc-b46a-46e9-aff5-23500341b18b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:56.912305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f354918-fe67-4b31-944d-e5693d8c24ca None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:56.913738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1191/2346] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:23:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:23:56.918826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:56.919689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:56.920618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:56.920919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:56.923806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249f9d5-c382-475e-a0ac-b650e78f0acd None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:56.924412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249f9d5-c382-475e-a0ac-b650e78f0acd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:56.924928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249f9d5-c382-475e-a0ac-b650e78f0acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:56.932668 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:56.934333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:56.937321 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:56.957582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7249f9d5-c382-475e-a0ac-b650e78f0acd None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:56.958855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7249f9d5-c382-475e-a0ac-b650e78f0acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:56.977299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1156/2347] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:56.978577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:57.000273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24832fb-b1bf-4eb2-96cf-1f5400fa248a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:57.013393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:57.014018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:57.015361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:57.017471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:57.018349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:57.020581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:57.155047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1157/2348] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:56 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:57.160474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:57.176896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1192/2349] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:23:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:57.256582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ffdbb-7864-4949-8b58-7c6fdaf80375 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:57.262120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:57.262848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:57.263229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:57.263918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:57.264779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:57.268162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:57.395525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1158/2350] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:57 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:57.396504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249f9d5-c382-475e-a0ac-b650e78f0acd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:57.410065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249f9d5-c382-475e-a0ac-b650e78f0acd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:57.415818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:57.416894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:57.417272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:57.417771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:57.418584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:57.420361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:57.432428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1193/2351] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:57.436814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:57.448101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bd161-9297-45fb-a414-2967eef26eb9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:57.458003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69469ee0-3613-45a6-a841-08d18418391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:57.458658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469ee0-3613-45a6-a841-08d18418391b admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:57.459259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469ee0-3613-45a6-a841-08d18418391b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:57.459447 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469ee0-3613-45a6-a841-08d18418391b admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:57.460263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69469ee0-3613-45a6-a841-08d18418391b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:57.462018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69469ee0-3613-45a6-a841-08d18418391b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:57.481206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1159/2352] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:57.485996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:57.501140 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998bd6c-cbb5-4b38-9f51-18b72d090162 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:57.518314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:57.519569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:57.520141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:57.520694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] PATH_INFO: `/v3/projects/64f73de1986f406182cc6a52b8376b93/users/8d42f70ad3cb4c9388c09a70cfc950dc/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:57.549014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f73de1986f406182cc6a52b8376b93, user_id=8d42f70ad3cb4c9388c09a70cfc950dc, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:57.552713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:57.666062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1194/2353] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:57 2022] PUT /identity/v3/projects/64f73de1986f406182cc6a52b8376b93/users/8d42f70ad3cb4c9388c09a70cfc950dc/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:57.675523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:57.675822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:57.676109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:57.714944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:57.716283 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:57.795220 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:57.819241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1160/2354] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:57.828414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:57.830780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:57.900144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fedd21-16b1-40a4-857a-6edd4b3340f5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:57.903768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d6b3e-50d1-4e96-9737-d9d233e862d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:57.904403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d6b3e-50d1-4e96-9737-d9d233e862d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:57.904693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d6b3e-50d1-4e96-9737-d9d233e862d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:57.904971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d6b3e-50d1-4e96-9737-d9d233e862d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:57.915019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:57.919384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d6b3e-50d1-4e96-9737-d9d233e862d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:57.921025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d6b3e-50d1-4e96-9737-d9d233e862d2 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:57.931612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1195/2355] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:06.692656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:06.694326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:06.773641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9371a-f1f8-45f3-9f24-aa8e2686d083 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:06.777629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40417e-0578-468c-91d9-6b39f27495d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:06.778352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40417e-0578-468c-91d9-6b39f27495d9 service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:06.778659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40417e-0578-468c-91d9-6b39f27495d9 service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:06.778975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40417e-0578-468c-91d9-6b39f27495d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:06.873811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40417e-0578-468c-91d9-6b39f27495d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:06.875423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40417e-0578-468c-91d9-6b39f27495d9 service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:06.901818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1161/2356] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:07.239961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c750b1f-967b-40c6-83a7-134e0539e8b1 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.240470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c750b1f-967b-40c6-83a7-134e0539e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.240553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c750b1f-967b-40c6-83a7-134e0539e8b1 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.242051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1196/2357] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:24:07.252913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eac1c-cccf-489b-a066-3e8bffe91cf0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.253236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eac1c-cccf-489b-a066-3e8bffe91cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.253487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eac1c-cccf-489b-a066-3e8bffe91cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.292678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-235eac1c-cccf-489b-a066-3e8bffe91cf0 None None] MFA Rules not processed for user `09cabb184e704aecbb0a3bfb1cd0a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:07.294551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-235eac1c-cccf-489b-a066-3e8bffe91cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:07.310639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1162/2358] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:24:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:07.326073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c750b1f-967b-40c6-83a7-134e0539e8b1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:07.347914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c750b1f-967b-40c6-83a7-134e0539e8b1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:07.357074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b8d97-885c-4036-9be1-19a2e79a0349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:07.357744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b8d97-885c-4036-9be1-19a2e79a0349 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.358028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b8d97-885c-4036-9be1-19a2e79a0349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.358308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b8d97-885c-4036-9be1-19a2e79a0349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.362601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a880c-81fa-48b8-898a-1d07681d8f4f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.363438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a880c-81fa-48b8-898a-1d07681d8f4f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.364349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a880c-81fa-48b8-898a-1d07681d8f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.371306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b8d97-885c-4036-9be1-19a2e79a0349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:07.372900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b8d97-885c-4036-9be1-19a2e79a0349 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:07.402039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1197/2359] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:07.449313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1c1a880c-81fa-48b8-898a-1d07681d8f4f None None] MFA Rules not processed for user `8d42f70ad3cb4c9388c09a70cfc950dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:07.456146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1c1a880c-81fa-48b8-898a-1d07681d8f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:07.560713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9976c-3332-4934-b0bd-2d33d171e18e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.561015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9976c-3332-4934-b0bd-2d33d171e18e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.561300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9976c-3332-4934-b0bd-2d33d171e18e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.562757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1198/2360] 10.222.0.49 () {54 vars in 921 bytes} [Tue Aug 2 10:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:24:07.563514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9976c-3332-4934-b0bd-2d33d171e18e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:07.566754 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1163/2361] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:24:07.580893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.581276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.581627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.583530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9976c-3332-4934-b0bd-2d33d171e18e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:07.589540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:07.590370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.590726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.591079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.591922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:07.611945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:07.662636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] MFA Rules not processed for user `9072775865604c73ae8361a047945464`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:07.668636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:07.691523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1164/2362] 10.222.0.49 () {58 vars in 1018 bytes} [Tue Aug 2 10:24:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:24:07.754329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:07.754329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:07.755510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147e75b-8541-4a79-abae-42cac7b2f63b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:07.759530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:07.760391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:07.762366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:07.762366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:07.770214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:07.776000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:07.777828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:07.795264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1165/2363] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:07.906198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1199/2364] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:24:07 2022] POST /identity/v3/projects => generated 371 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:07.910676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:07.939678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44139a-a306-4ae6-9e97-4e05ba6ab1ca service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:08.036241 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:08.036985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:08.037358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:08.038189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:08.038841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:08.041277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:08.204749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1166/2365] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:24:07 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:08.208974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:08.231335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2b7d5-e70f-4fc7-a562-e880199a2fcd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:08.240862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:08.240862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:08.240862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:08.240862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:08.241701 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:08.242473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:08.255426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1200/2366] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:08.260573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:08.271714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda78b55-bf4d-4e08-aee8-7f2d325ed5ee admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:08.277072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:08.278213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:08.278706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:08.280065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] PATH_INFO: `/v3/projects/340a38fa81ce46b6be226b1f7a2f8a3b/users/c564373bad3344be9c8d58d28276681d/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:08.296407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340a38fa81ce46b6be226b1f7a2f8a3b, user_id=c564373bad3344be9c8d58d28276681d, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:08.299784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:08.400298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1167/2367] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:24:08 2022] PUT /identity/v3/projects/340a38fa81ce46b6be226b1f7a2f8a3b/users/c564373bad3344be9c8d58d28276681d/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:08.402371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:08.414733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cdb2e-8f64-4e6c-a267-dd7361de2bdf admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:08.480442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:08.481402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:08.481694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:08.484766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:08.484766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:08.484766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:08.495313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1201/2368] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:08.499596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:08.523710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b91912-cb9b-4156-9a35-46de2ba8b713 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:08.539004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:08.539901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:08.540204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:08.540558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] PATH_INFO: `/v3/projects/340a38fa81ce46b6be226b1f7a2f8a3b/users/c564373bad3344be9c8d58d28276681d/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:08.542784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=340a38fa81ce46b6be226b1f7a2f8a3b, user_id=c564373bad3344be9c8d58d28276681d, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:08.545738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:08.557395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1168/2369] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:24:08 2022] PUT /identity/v3/projects/340a38fa81ce46b6be226b1f7a2f8a3b/users/c564373bad3344be9c8d58d28276681d/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:24:13.822614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:13.824756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:13.960626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab4cc7-9ee0-4e4f-95d3-b0814d289325 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:14.029531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:14.052825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762cf8f-185b-43f9-afad-804b8566f7bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:14.053507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762cf8f-185b-43f9-afad-804b8566f7bd service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:14.053743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762cf8f-185b-43f9-afad-804b8566f7bd service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:14.054022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762cf8f-185b-43f9-afad-804b8566f7bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:14.055894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5762cf8f-185b-43f9-afad-804b8566f7bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:14.071078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762cf8f-185b-43f9-afad-804b8566f7bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:14.073183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762cf8f-185b-43f9-afad-804b8566f7bd service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:14.091125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1202/2370] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:15.491722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:15.498517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:15.606009 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad433-51dd-4f2b-82d1-5eb21b945aee admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:15.611360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:15.612092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:15.612289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:15.612581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] PATH_INFO: `/v3/users/37bf163d0df648b6a39f29c0697537f8` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:15.613802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37bf163d0df648b6a39f29c0697537f8)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:15.617057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:15.972404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37bf163d0df648b6a39f29c0697537f8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:15.973108 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37bf163d0df648b6a39f29c0697537f8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:15.974777 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:15.992273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37bf163d0df648b6a39f29c0697537f8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:16.116620 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37bf163d0df648b6a39f29c0697537f8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:16.159270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37bf163d0df648b6a39f29c0697537f8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:16.182340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1169/2371] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:24:15 2022] DELETE /identity/v3/users/37bf163d0df648b6a39f29c0697537f8 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:16.186677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:16.187325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:16.187325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:16.226181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:16.227561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:16.338657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1203/2372] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:16.361417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:16.362794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:16.512089 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab39ee2-6583-422a-9a94-e934d1e343e5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:16.516464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab54d4-9439-420b-9192-ae0121045fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:16.517794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab54d4-9439-420b-9192-ae0121045fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:16.518206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab54d4-9439-420b-9192-ae0121045fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:16.518510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab54d4-9439-420b-9192-ae0121045fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:16.528896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab54d4-9439-420b-9192-ae0121045fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:16.530518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab54d4-9439-420b-9192-ae0121045fb5 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:16.547240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1170/2373] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:17.237079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:17.238670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:17.265636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aa9f3-d3ec-4d5f-9df7-2ed7d8166a19 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:17.278625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:17.279584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:17.280102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:17.280565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] PATH_INFO: `/v3/projects/b63a11e1f15d4536ba7d875abc5bb7b4` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:17.282926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63a11e1f15d4536ba7d875abc5bb7b4)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:17.290984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:17.498399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63a11e1f15d4536ba7d875abc5bb7b4', 'request_id': 'req-2c72036d-6678-40ac-9415-18c4b87e0978'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:17.627969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63a11e1f15d4536ba7d875abc5bb7b4', 'request_id': 'req-2c72036d-6678-40ac-9415-18c4b87e0978'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:17.638879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63a11e1f15d4536ba7d875abc5bb7b4', 'request_id': 'req-2c72036d-6678-40ac-9415-18c4b87e0978'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:17.657875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] The token cache is being invalidate because project b63a11e1f15d4536ba7d875abc5bb7b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:24:17.658620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:17.659659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c72036d-6678-40ac-9415-18c4b87e0978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:17.661573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1204/2374] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:24:17 2022] DELETE /identity/v3/projects/b63a11e1f15d4536ba7d875abc5bb7b4 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:17.667137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017525-0609-4dc5-b498-0f91be009b03 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:17.667734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017525-0609-4dc5-b498-0f91be009b03 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:17.667734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017525-0609-4dc5-b498-0f91be009b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:17.689558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-31017525-0609-4dc5-b498-0f91be009b03 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:17.690785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-31017525-0609-4dc5-b498-0f91be009b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:17.761667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1171/2375] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:17.770985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afc705-c99a-4d84-af0a-cbd4b264514f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:17.771703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afc705-c99a-4d84-af0a-cbd4b264514f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:17.777331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afc705-c99a-4d84-af0a-cbd4b264514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:17.795162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-14afc705-c99a-4d84-af0a-cbd4b264514f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:17.796963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-14afc705-c99a-4d84-af0a-cbd4b264514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:17.813159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1205/2376] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:17.815562 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31017525-0609-4dc5-b498-0f91be009b03 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:17.833356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31017525-0609-4dc5-b498-0f91be009b03 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:17.839087 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:17.840039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:17.840409 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:17.840694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:17.841417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:24:17.841876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:17.843904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:17.861118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1172/2377] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:24:17.865404 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afc705-c99a-4d84-af0a-cbd4b264514f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:17.880561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afc705-c99a-4d84-af0a-cbd4b264514f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:17.890174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:17.890831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:17.891314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:17.892193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:17.893038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:17.895062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:17.999532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1206/2378] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:24:17 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:18.005039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:18.017189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f5e9f-6374-4051-b24f-3a502c994485 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:18.100923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:18.108503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:18.109174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:18.109461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:18.110784 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:18.111007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:18.113155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:18.264612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1173/2379] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:24:18 2022] POST /identity/v3/users => generated 292 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:18.266116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:18.281799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f5f51-369c-487a-8e76-7b7dca342f35 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:18.304953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:18.306649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:18.308592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:18.309284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:18.310126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:18.312256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:18.327917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1207/2380] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:18.332648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:18.357311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c5a31-c36a-4293-bdc2-e8a646dde798 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:18.373322 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12345d-99e8-4481-a382-890f9c1209f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:18.374083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12345d-99e8-4481-a382-890f9c1209f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:18.374515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12345d-99e8-4481-a382-890f9c1209f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:18.374879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12345d-99e8-4481-a382-890f9c1209f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:18.375722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12345d-99e8-4481-a382-890f9c1209f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:18.378073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12345d-99e8-4481-a382-890f9c1209f1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:18.395338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1174/2381] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:18.399971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:18.424879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df53bad-f8c5-482c-b0ec-dd481344dea1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:18.430285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:18.431117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:18.433433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:18.433988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] PATH_INFO: `/v3/projects/1de53055b1004d179d351bc2cb50c86b/users/e35dc9a416054640822974ab8e1882ba/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:18.465859 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de53055b1004d179d351bc2cb50c86b, user_id=e35dc9a416054640822974ab8e1882ba, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:18.470287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:18.474891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:18.560023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1208/2382] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:24:18 2022] PUT /identity/v3/projects/1de53055b1004d179d351bc2cb50c86b/users/e35dc9a416054640822974ab8e1882ba/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:18.573502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35897-2323-4c0e-939e-bbdb3e30c361 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:18.578196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35897-2323-4c0e-939e-bbdb3e30c361 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:18.578196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35897-2323-4c0e-939e-bbdb3e30c361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:18.637688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3cd35897-2323-4c0e-939e-bbdb3e30c361 None None] MFA Rules not processed for user `e35dc9a416054640822974ab8e1882ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:18.638652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3cd35897-2323-4c0e-939e-bbdb3e30c361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:18.726367 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1175/2383] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:18.730746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:18.828354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08ab82-d327-429c-b467-fa3c10421181 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:18.834858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:18.836506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:18.837114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:18.838145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:18.843669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:18.849570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:18.863789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:18.864965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:18.866000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:18.941132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] MFA Rules not processed for user `c564373bad3344be9c8d58d28276681d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:18.949536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:18.955152 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1209/2384] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:24:18 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:18.956776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:19.087530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d3f1f-4f13-4adc-9753-dac00c0e7e6b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:19.102589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:19.104132 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.104594 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.106088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.106517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:19.109697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:19.215673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1176/2385] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:19.223112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:19.224843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:19.249309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1210/2386] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:24:18 2022] POST /identity/v3/users => generated 293 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:24:19.253542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:19.301090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f48f6-58d8-4dd3-86df-6a67c0215090 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:19.333665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:19.335628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.338304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.339432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.347810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:19.349812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:19.357125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:19.387159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1211/2387] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:24:19.391231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:19.391919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fc8c2-597f-42bf-9e10-7d661d21e74c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:19.399021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277be618-9ed3-4061-8527-4a33d2dc1d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:19.399707 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277be618-9ed3-4061-8527-4a33d2dc1d5a service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.399996 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277be618-9ed3-4061-8527-4a33d2dc1d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.400280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277be618-9ed3-4061-8527-4a33d2dc1d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.403603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-277be618-9ed3-4061-8527-4a33d2dc1d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:19.415589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840e33d-036f-4300-90b1-effc2cc60a3d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:19.429322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:19.430278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.431892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.432349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.433235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:19.443605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:19.472876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1212/2388] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:19.475150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:19.496528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b62e2-60fd-4648-905a-f118b0b55dbf admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:19.511762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25049b-a8a9-4225-9ae9-b98c1664e4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:19.512417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25049b-a8a9-4225-9ae9-b98c1664e4d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.512704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25049b-a8a9-4225-9ae9-b98c1664e4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.513255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25049b-a8a9-4225-9ae9-b98c1664e4d9 admin admin] PATH_INFO: `/v3/projects/7625343f0e164e86bb99dd1c89acea3d/users/d18dfdd1ceda43bc9954348fb6ddccc3/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.553138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277be618-9ed3-4061-8527-4a33d2dc1d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:19.554881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277be618-9ed3-4061-8527-4a33d2dc1d5a service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:19.564687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25049b-a8a9-4225-9ae9-b98c1664e4d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7625343f0e164e86bb99dd1c89acea3d, user_id=d18dfdd1ceda43bc9954348fb6ddccc3, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:19.566612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1177/2389] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:24:19.569557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25049b-a8a9-4225-9ae9-b98c1664e4d9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:19.672108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1213/2390] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:24:19 2022] PUT /identity/v3/projects/7625343f0e164e86bb99dd1c89acea3d/users/d18dfdd1ceda43bc9954348fb6ddccc3/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:19.681521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d066-43d9-41d3-b7fc-1c01a12b25fb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.681822 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d066-43d9-41d3-b7fc-1c01a12b25fb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.682267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6d066-43d9-41d3-b7fc-1c01a12b25fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.747597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b8f6d066-43d9-41d3-b7fc-1c01a12b25fb None None] MFA Rules not processed for user `d18dfdd1ceda43bc9954348fb6ddccc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:19.749684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b8f6d066-43d9-41d3-b7fc-1c01a12b25fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:19.760104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.760707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.761195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.791060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:19.792464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:19.814194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:19.839800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1178/2391] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:19.844475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d066-43d9-41d3-b7fc-1c01a12b25fb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:19.858614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1214/2392] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:24:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:24:19.877309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:19.878792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:19.940603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6d066-43d9-41d3-b7fc-1c01a12b25fb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:19.946518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:19.947253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.947515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.947798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.948475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:19.950529 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:19.963247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0996d-8f24-42d1-8c9a-27b564e9aeac None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:19.971258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:19.972073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:19.972414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:19.972747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:19.983991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:19.986195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:19.996575 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1215/2393] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:24:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:24:20.060558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:20.084827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:20.103856 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1179/2394] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:24:19 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:20.107618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2341-b802-406f-8f1a-97c0f22e8142 service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:20.303094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:20.304609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:20.304896 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:20.305376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:20.306061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:20.308045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:20.682998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1216/2395] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:24:20 2022] POST /identity/v3/users => generated 299 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:20.683486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:20.698488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250d1fe-4183-431c-be7e-acfa87b1ad17 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:20.701151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:20.701879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:20.702266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:20.702757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:20.703612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:20.705633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:20.725512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1180/2396] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:20.729639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:20.747841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a035c-9cb7-4040-90d7-0af10e2c1cb9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:20.753962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:20.754644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:20.755091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:20.756308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] PATH_INFO: `/v3/projects/fb3e07563ede49e1b9b499fea641c419/users/3b77d1c642d441b8a5baeafb3a7490c3/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:20.772930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3e07563ede49e1b9b499fea641c419, user_id=3b77d1c642d441b8a5baeafb3a7490c3, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:20.779066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:20.923447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1217/2397] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:24:20 2022] PUT /identity/v3/projects/fb3e07563ede49e1b9b499fea641c419/users/3b77d1c642d441b8a5baeafb3a7490c3/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:24:20.928428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:20.942731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de487960-68e3-4327-8b26-7d142332ff89 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:21.051437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45aa10-ba52-47cd-8cf7-72cfd687aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:21.052373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45aa10-ba52-47cd-8cf7-72cfd687aac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:21.052861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45aa10-ba52-47cd-8cf7-72cfd687aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:21.053548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45aa10-ba52-47cd-8cf7-72cfd687aac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:21.054439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45aa10-ba52-47cd-8cf7-72cfd687aac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:21.078052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45aa10-ba52-47cd-8cf7-72cfd687aac0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:21.098720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1181/2398] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:21.099340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:21.119565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64b92f-00b7-4b4e-80a6-51cb4dcacab3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:21.125445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:21.128251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:21.128604 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:21.129222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] PATH_INFO: `/v3/projects/fb3e07563ede49e1b9b499fea641c419/users/3b77d1c642d441b8a5baeafb3a7490c3/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:21.131779 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3e07563ede49e1b9b499fea641c419, user_id=3b77d1c642d441b8a5baeafb3a7490c3, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:21.135429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:21.151645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1218/2399] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:24:21 2022] PUT /identity/v3/projects/fb3e07563ede49e1b9b499fea641c419/users/3b77d1c642d441b8a5baeafb3a7490c3/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:24:21.163153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b99efd-d015-4091-9c4f-1921ec4eed08 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:21.163557 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b99efd-d015-4091-9c4f-1921ec4eed08 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:21.167238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b99efd-d015-4091-9c4f-1921ec4eed08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:21.237982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-93b99efd-d015-4091-9c4f-1921ec4eed08 None None] MFA Rules not processed for user `3b77d1c642d441b8a5baeafb3a7490c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:21.255905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-93b99efd-d015-4091-9c4f-1921ec4eed08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:21.411640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1182/2400] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:21.427762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:21.429679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:21.481659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:21.601806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e371e5-f2e5-4637-b36c-b3b29a09bff7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:21.614322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:21.620880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:21.621225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:21.622512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:21.744384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:21.746857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:21.773951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1219/2401] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:23.842661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275d52c-487d-42bd-9af0-7994c1bdf459 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:23.847989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275d52c-487d-42bd-9af0-7994c1bdf459 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:23.847989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275d52c-487d-42bd-9af0-7994c1bdf459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:23.939489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4275d52c-487d-42bd-9af0-7994c1bdf459 None None] MFA Rules not processed for user `8d42f70ad3cb4c9388c09a70cfc950dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:23.941415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4275d52c-487d-42bd-9af0-7994c1bdf459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:24.123589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1183/2402] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:24.500334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:24.503069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:24.518521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd4f6-6c4b-49c3-8097-133a69811478 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:24.525326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965391ed-2842-4ff4-91d6-22b22b77532d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:24.525326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965391ed-2842-4ff4-91d6-22b22b77532d service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:24.525326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965391ed-2842-4ff4-91d6-22b22b77532d service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:24.525326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965391ed-2842-4ff4-91d6-22b22b77532d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:24.636186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965391ed-2842-4ff4-91d6-22b22b77532d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:24.665406 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965391ed-2842-4ff4-91d6-22b22b77532d service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:24.678293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1220/2403] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:27.145863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275d52c-487d-42bd-9af0-7994c1bdf459 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:27.170448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275d52c-487d-42bd-9af0-7994c1bdf459 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:27.180105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:27.181039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:27.181462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:27.181894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:27.198339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:27.199933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:27.212311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1184/2404] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:30.745030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0580-4088-4d7b-80f3-2a1ad5f210e7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:30.745619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0580-4088-4d7b-80f3-2a1ad5f210e7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:30.745619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0580-4088-4d7b-80f3-2a1ad5f210e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:30.765424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2c9a0580-4088-4d7b-80f3-2a1ad5f210e7 None None] MFA Rules not processed for user `1a800d95ac1d492bb44947fdf5bb3377`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:30.767391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2c9a0580-4088-4d7b-80f3-2a1ad5f210e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:30.822423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1221/2405] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:35.228673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:35.233160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:35.337821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df056672-29f0-4b7b-9f2a-c977e2c70ed4 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:35.339723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:35.342079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:35.342551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:35.342924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] PATH_INFO: `/v3/users/e35dc9a416054640822974ab8e1882ba` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:35.344334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35dc9a416054640822974ab8e1882ba)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:35.357240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:35.720240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35dc9a416054640822974ab8e1882ba'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:35.720950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35dc9a416054640822974ab8e1882ba'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:35.728558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35dc9a416054640822974ab8e1882ba'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:35.818088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35dc9a416054640822974ab8e1882ba'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:35.832178 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35dc9a416054640822974ab8e1882ba'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:35.838525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1185/2406] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:24:35 2022] DELETE /identity/v3/users/e35dc9a416054640822974ab8e1882ba => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:35.843088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:35.843745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:35.843900 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:35.864720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:35.866246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:35.870666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:35.930634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1222/2407] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:35.960731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:35.962558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:36.060688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b1ab-ebba-4345-af63-f5b5649c6797 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:36.066201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:36.066999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:36.067347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:36.068410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:36.077217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:36.078678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:36.092137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1186/2408] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:36.809897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:36.811500 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:36.823910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966bd1b-13f7-43b0-b019-21886a416313 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:36.830331 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:36.830975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:36.831313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:36.831602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] PATH_INFO: `/v3/projects/1de53055b1004d179d351bc2cb50c86b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:36.837067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de53055b1004d179d351bc2cb50c86b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:36.837067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:36.988517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de53055b1004d179d351bc2cb50c86b', 'request_id': 'req-52c3786b-66db-4930-a07d-faf8e4948155'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.075854 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de53055b1004d179d351bc2cb50c86b', 'request_id': 'req-52c3786b-66db-4930-a07d-faf8e4948155'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.082685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de53055b1004d179d351bc2cb50c86b', 'request_id': 'req-52c3786b-66db-4930-a07d-faf8e4948155'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.098286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] The token cache is being invalidate because project 1de53055b1004d179d351bc2cb50c86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:24:37.098490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.099172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.101739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1223/2409] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:24:36 2022] DELETE /identity/v3/projects/1de53055b1004d179d351bc2cb50c86b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:37.106458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:37.120344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5c678-2b50-416b-a139-c6d93a2d8830 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:37.140954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:37.180762 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:37.181420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:37.181703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:37.182022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] PATH_INFO: `/v3/users/d18dfdd1ceda43bc9954348fb6ddccc3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:37.183305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18dfdd1ceda43bc9954348fb6ddccc3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:37.185242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:37.467705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18dfdd1ceda43bc9954348fb6ddccc3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.468402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18dfdd1ceda43bc9954348fb6ddccc3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.477436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18dfdd1ceda43bc9954348fb6ddccc3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.596267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18dfdd1ceda43bc9954348fb6ddccc3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.609219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18dfdd1ceda43bc9954348fb6ddccc3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:37.616714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1187/2410] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:24:37 2022] DELETE /identity/v3/users/d18dfdd1ceda43bc9954348fb6ddccc3 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:38.128084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:38.129697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:38.153975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3786b-66db-4930-a07d-faf8e4948155 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:38.218238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:38.218893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:38.219228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:38.219531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] PATH_INFO: `/v3/projects/7625343f0e164e86bb99dd1c89acea3d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:38.221283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7625343f0e164e86bb99dd1c89acea3d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:38.223296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:38.348816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:38.360281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7625343f0e164e86bb99dd1c89acea3d', 'request_id': 'req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.464136 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7625343f0e164e86bb99dd1c89acea3d', 'request_id': 'req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.470994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7625343f0e164e86bb99dd1c89acea3d', 'request_id': 'req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.484501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] The token cache is being invalidate because project 7625343f0e164e86bb99dd1c89acea3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:24:38.484800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.485468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.490496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1224/2411] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:24:38 2022] DELETE /identity/v3/projects/7625343f0e164e86bb99dd1c89acea3d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:38.491914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:38.493384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:38.506602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5193d12-4687-4167-8e5c-e1693ce08640 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:38.577558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:38.578145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:38.578434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:38.578712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] PATH_INFO: `/v3/users/3b77d1c642d441b8a5baeafb3a7490c3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:38.580364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b77d1c642d441b8a5baeafb3a7490c3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:38.582558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:38.785719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:38.790342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b77d1c642d441b8a5baeafb3a7490c3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.791018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b77d1c642d441b8a5baeafb3a7490c3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.798997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b77d1c642d441b8a5baeafb3a7490c3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.859698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:38.861808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:38.875291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b77d1c642d441b8a5baeafb3a7490c3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.878027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f780-7cf3-4f5a-91ef-c83e1f39e2aa admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:38.886824 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:38.887629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:38.887986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:38.888357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] PATH_INFO: `/v3/users/5500c79a37db423e8812128a1d0314f8` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:38.889839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5500c79a37db423e8812128a1d0314f8)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:38.892278 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:38.897103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-969d40e6-659a-42d3-9486-3c2d777a80ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b77d1c642d441b8a5baeafb3a7490c3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:38.903774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1188/2412] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:24:38 2022] DELETE /identity/v3/users/3b77d1c642d441b8a5baeafb3a7490c3 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:39.112508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5500c79a37db423e8812128a1d0314f8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.119670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5500c79a37db423e8812128a1d0314f8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.120370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5500c79a37db423e8812128a1d0314f8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.120975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5500c79a37db423e8812128a1d0314f8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.146963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5500c79a37db423e8812128a1d0314f8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.235637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1225/2413] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:24:38 2022] DELETE /identity/v3/users/5500c79a37db423e8812128a1d0314f8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:24:39.241506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa312-7961-417b-b803-87295201b2bb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:39.242864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa312-7961-417b-b803-87295201b2bb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:39.242864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa312-7961-417b-b803-87295201b2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:39.265127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-de0fa312-7961-417b-b803-87295201b2bb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:39.269630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-de0fa312-7961-417b-b803-87295201b2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:39.358078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1189/2414] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:39.368437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:39.370003 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:39.469159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa312-7961-417b-b803-87295201b2bb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:39.470639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-de0fa312-7961-417b-b803-87295201b2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:39.475595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad213-c0dc-48c4-a9fb-def54e566df9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:39.481290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd6dde-bd1c-49bb-b7c3-94f75ba725cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:39.482359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd6dde-bd1c-49bb-b7c3-94f75ba725cf service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:39.482839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd6dde-bd1c-49bb-b7c3-94f75ba725cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:39.483367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd6dde-bd1c-49bb-b7c3-94f75ba725cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:39.492444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa312-7961-417b-b803-87295201b2bb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:39.500430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:39.501115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:39.501919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:39.502227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd6dde-bd1c-49bb-b7c3-94f75ba725cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:39.502649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] PATH_INFO: `/v3/projects/fb3e07563ede49e1b9b499fea641c419` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:39.504076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd6dde-bd1c-49bb-b7c3-94f75ba725cf service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:39.505422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3e07563ede49e1b9b499fea641c419)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:39.513918 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:39.518153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1226/2415] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:39.646573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3e07563ede49e1b9b499fea641c419', 'request_id': 'req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.657055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3e07563ede49e1b9b499fea641c419', 'request_id': 'req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.761525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3e07563ede49e1b9b499fea641c419', 'request_id': 'req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.786034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] The token cache is being invalidate because project fb3e07563ede49e1b9b499fea641c419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:24:39.786560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.787564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba8e3a2d-6e22-4cfd-b2e9-7a6f6555aa5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:39.795170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1190/2416] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:24:39 2022] DELETE /identity/v3/projects/fb3e07563ede49e1b9b499fea641c419 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:39.800678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9ae0-eff8-4383-862b-830939de60e6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:39.804739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9ae0-eff8-4383-862b-830939de60e6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:39.804739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9ae0-eff8-4383-862b-830939de60e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:39.827616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-81db9ae0-eff8-4383-862b-830939de60e6 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:39.829411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-81db9ae0-eff8-4383-862b-830939de60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:39.944763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:39.979567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1227/2417] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:39.981264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144600f-0db3-47da-beec-0dd662cddb12 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:39.981550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144600f-0db3-47da-beec-0dd662cddb12 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:39.981977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144600f-0db3-47da-beec-0dd662cddb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.021032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a144600f-0db3-47da-beec-0dd662cddb12 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:40.022400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a144600f-0db3-47da-beec-0dd662cddb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:40.040191 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1191/2418] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:24:40.044387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db9ae0-eff8-4383-862b-830939de60e6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.054791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db9ae0-eff8-4383-862b-830939de60e6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.059572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.060194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.060478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.060752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.061455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:24:40.061826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:40.063501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:40.079082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1228/2419] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:24:40.083660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144600f-0db3-47da-beec-0dd662cddb12 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.093943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144600f-0db3-47da-beec-0dd662cddb12 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.103410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.104055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.104349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.104629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.105311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:40.107285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:40.120671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.121628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:40.136898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc180fb-e3a5-4fc3-a9f8-5e0e4e1c2d6b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.148662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.148662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.148662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.148662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] PATH_INFO: `/v3/projects/74e053ef85664c65928a62b9c54a9f9e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.150747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e053ef85664c65928a62b9c54a9f9e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:40.152853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:40.233883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1192/2420] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:24:40 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:24:40.237420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.283901 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:40.299795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e053ef85664c65928a62b9c54a9f9e', 'request_id': 'req-f1386203-e94b-433f-a96b-e727d4cb7c0e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:40.350840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eed40a-99fd-491a-a19d-bd97d010fe1d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.378185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e053ef85664c65928a62b9c54a9f9e', 'request_id': 'req-f1386203-e94b-433f-a96b-e727d4cb7c0e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:40.385496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e053ef85664c65928a62b9c54a9f9e', 'request_id': 'req-f1386203-e94b-433f-a96b-e727d4cb7c0e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:40.404650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] The token cache is being invalidate because project 74e053ef85664c65928a62b9c54a9f9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:24:40.404936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:40.405809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:24:40.408526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1229/2421] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:24:40 2022] DELETE /identity/v3/projects/74e053ef85664c65928a62b9c54a9f9e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:40.429692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.430594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.431382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.431672 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.432363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:40.434576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:40.573561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1193/2422] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:24:40 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:24:40.577784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.579443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:40.603536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1386203-e94b-433f-a96b-e727d4cb7c0e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.612315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.612315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.612315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.612315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.612930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:40.615533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:40.628170 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1230/2423] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:40.631872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.644239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a8333-a520-4776-88bc-f7bb3bbeb8b5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.648904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee86c54-d7f5-4353-b410-c76ce5e65ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.649522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee86c54-d7f5-4353-b410-c76ce5e65ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.649834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee86c54-d7f5-4353-b410-c76ce5e65ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.650135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee86c54-d7f5-4353-b410-c76ce5e65ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.650891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee86c54-d7f5-4353-b410-c76ce5e65ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:40.652555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee86c54-d7f5-4353-b410-c76ce5e65ec8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:40.663308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1194/2424] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:24:40.667563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.677963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e95add-011d-4897-846f-6aa1a11c5815 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.683172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.683907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.684211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.684478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] PATH_INFO: `/v3/projects/c8fc70abe3344813a5c0b831b68f7c71/users/e7f1b0932111477caa21cd18270cd8c5/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.700504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8fc70abe3344813a5c0b831b68f7c71, user_id=e7f1b0932111477caa21cd18270cd8c5, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:40.703362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:40.786065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1231/2425] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:24:40 2022] PUT /identity/v3/projects/c8fc70abe3344813a5c0b831b68f7c71/users/e7f1b0932111477caa21cd18270cd8c5/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:24:40.796319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0287bc-c4b4-4a22-a611-94c9cc9dd6c1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.796544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0287bc-c4b4-4a22-a611-94c9cc9dd6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.796834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0287bc-c4b4-4a22-a611-94c9cc9dd6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.825212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2f0287bc-c4b4-4a22-a611-94c9cc9dd6c1 None None] MFA Rules not processed for user `e7f1b0932111477caa21cd18270cd8c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:40.826435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2f0287bc-c4b4-4a22-a611-94c9cc9dd6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:40.899953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1195/2426] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:40.912255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:40.917517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:40.975807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f459c7-237d-4be4-8a1a-50b5199f6db4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:40.979877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae297-9734-4738-a2f7-8e92e1ff019e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:40.980183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae297-9734-4738-a2f7-8e92e1ff019e service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:40.980465 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae297-9734-4738-a2f7-8e92e1ff019e service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:40.980743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae297-9734-4738-a2f7-8e92e1ff019e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:40.990034 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae297-9734-4738-a2f7-8e92e1ff019e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:41.000757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae297-9734-4738-a2f7-8e92e1ff019e service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:41.001903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1232/2427] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:42.898561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:42.899103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:42.899103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:42.918492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] MFA Rules not processed for user `e7f1b0932111477caa21cd18270cd8c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:42.919790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:42.927841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1196/2428] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:45.675228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:45.676045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:45.676717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:45.697793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:45.699635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:45.710895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:45.785256 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1233/2429] 10.222.0.49 () {58 vars in 967 bytes} [Tue Aug 2 10:24:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:24:45.821306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:45.823034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:45.872445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:45.966680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d0f2-98d6-4600-81a2-5534c808113d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:45.972329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1866f9-9078-4250-83d8-0ee965166b55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:45.973200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1866f9-9078-4250-83d8-0ee965166b55 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:45.973491 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1866f9-9078-4250-83d8-0ee965166b55 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:45.973772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1866f9-9078-4250-83d8-0ee965166b55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:45.984394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1866f9-9078-4250-83d8-0ee965166b55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:45.986904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1866f9-9078-4250-83d8-0ee965166b55 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:45.996333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1197/2430] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:24:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:24:53.593088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:24:53.594626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:53.653979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b06b1-f7ef-4bde-9c0c-5e8078f0048d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:24:53.657731 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6809e3-53eb-4608-bd6b-49ad9888dfd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:24:53.658361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6809e3-53eb-4608-bd6b-49ad9888dfd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:53.658703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6809e3-53eb-4608-bd6b-49ad9888dfd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:53.659506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6809e3-53eb-4608-bd6b-49ad9888dfd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:53.661287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aa6809e3-53eb-4608-bd6b-49ad9888dfd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:53.736883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6809e3-53eb-4608-bd6b-49ad9888dfd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:24:53.738675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6809e3-53eb-4608-bd6b-49ad9888dfd4 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:24:53.751698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1234/2431] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:54.884394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1952f6-9771-480a-ad10-15ef9673d0ae None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:24:54.884940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1952f6-9771-480a-ad10-15ef9673d0ae None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:24:54.884940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1952f6-9771-480a-ad10-15ef9673d0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:24:54.914845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-da1952f6-9771-480a-ad10-15ef9673d0ae None None] MFA Rules not processed for user `9e50546af342492b9b6f6127b0ce2c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:24:54.916201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-da1952f6-9771-480a-ad10-15ef9673d0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:24:54.982965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1198/2432] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:24:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:25:03.757035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=68 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:04.975099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=45 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:06.773949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757110a8-1504-423b-9567-95bde86f654f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.774847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757110a8-1504-423b-9567-95bde86f654f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.775633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757110a8-1504-423b-9567-95bde86f654f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.798422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-757110a8-1504-423b-9567-95bde86f654f None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:06.799737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-757110a8-1504-423b-9567-95bde86f654f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:06.807135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1235/2433] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:06.961155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1952f6-9771-480a-ad10-15ef9673d0ae None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:06.973483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1952f6-9771-480a-ad10-15ef9673d0ae None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:06.979271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:06.979939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:06.980159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:06.980455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:06.988458 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:06.990040 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:06.998266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1199/2434] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:07.149044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757110a8-1504-423b-9567-95bde86f654f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:07.159349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757110a8-1504-423b-9567-95bde86f654f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:07.162979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25667ac-6242-4fe4-bd87-c3db54ecc3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:07.163623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25667ac-6242-4fe4-bd87-c3db54ecc3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:07.163907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25667ac-6242-4fe4-bd87-c3db54ecc3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:07.164243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25667ac-6242-4fe4-bd87-c3db54ecc3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:07.166010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d25667ac-6242-4fe4-bd87-c3db54ecc3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:07.257673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25667ac-6242-4fe4-bd87-c3db54ecc3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:07.260618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25667ac-6242-4fe4-bd87-c3db54ecc3f4 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:07.272460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1236/2435] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:07.520797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:07.523115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:07.536731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4aa05-3bd1-4fe4-9d82-b44a5fdeab6f service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:07.541038 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:07.541744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:07.542031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:07.542312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:07.544173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:07.574068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:07.575681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:07.588819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1200/2436] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:08.062090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d02357-1ad3-4745-a7bb-2a27964021d6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.062589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d02357-1ad3-4745-a7bb-2a27964021d6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.062684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d02357-1ad3-4745-a7bb-2a27964021d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.082525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-26d02357-1ad3-4745-a7bb-2a27964021d6 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:08.083750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-26d02357-1ad3-4745-a7bb-2a27964021d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:08.091375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1237/2437] 10.222.0.49 () {58 vars in 967 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:08.103913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:08.115386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee834b4b-20bf-4582-b8fa-67cbbb66332b service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:08.121534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:08.122156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.122626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.122908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.131413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:08.133006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:08.141914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1201/2438] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:08.968129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00fbb-33c9-4b5a-ac01-a8dc680ed156 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:08.968609 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00fbb-33c9-4b5a-ac01-a8dc680ed156 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:08.969044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00fbb-33c9-4b5a-ac01-a8dc680ed156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:08.995746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a0e00fbb-33c9-4b5a-ac01-a8dc680ed156 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:08.995746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a0e00fbb-33c9-4b5a-ac01-a8dc680ed156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:09.007358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1238/2439] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:09.023028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:09.034889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01189b-96ee-4c63-8198-f4e72b6eee9d service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:09.045745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:09.046667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.047150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.047465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.058721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:09.061286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:09.070812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1202/2440] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:09.784790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e53e1b-511d-4c51-9052-063c02a66eb0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.785676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e53e1b-511d-4c51-9052-063c02a66eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.786848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e53e1b-511d-4c51-9052-063c02a66eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.810010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-39e53e1b-511d-4c51-9052-063c02a66eb0 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:26:09.811550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-39e53e1b-511d-4c51-9052-063c02a66eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:09.825855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1239/2441] 10.222.0.49 () {58 vars in 967 bytes} [Tue Aug 2 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:26:09.840165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:09.891053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616db72-dd23-42da-b26b-f3a691b6c6d9 service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:09.891053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:09.891053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:09.891053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:09.892649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:09.892649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:09.892649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:09.892649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1203/2442] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:26:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:26:12.908953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e53e1b-511d-4c51-9052-063c02a66eb0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:12.919535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e53e1b-511d-4c51-9052-063c02a66eb0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:12.923102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:12.923761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:12.924047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:12.924326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:12.926098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:12.993125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:12.994820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:13.004672 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1240/2443] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:15.279899 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:15.281339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:15.297183 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8f899-5333-4645-8c58-096711e3a076 service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:15.304181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:15.305010 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:15.305284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:15.305716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:15.308162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:15.391012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:15.392770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:15.403865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1204/2444] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:19.440987 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:19.453837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db2b64-8b8e-4f81-a139-e73da15e9589 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:19.459179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:19.459805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:19.460086 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:19.460362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:19.462100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:19.513583 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:19.523639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:19.525231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:19.534639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1241/2445] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:25.398984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=58 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:29.557602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:01.267146 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:01.281477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59bfc-b0a1-43a6-87e7-cf566781cb1e service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:01.293300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:01.294378 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:01.294988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:01.295801 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:01.300516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:01.322944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:01.326851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:01.356148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1205/2446] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:01.437503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:01.464780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f509cb7-7651-4d1f-99c6-c5c7b0c33410 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:01.469112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:01.469869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:01.470221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:01.470599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:01.472884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:01.483915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:01.485463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:01.495303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1242/2447] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:11.349192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:11.491342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:41.221041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:41.235251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab94690-bf86-4d15-8c5a-386ad4a21e37 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:41.235251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:41.235748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:41.235834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:41.236060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:41.237954 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:41.299433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:41.300931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:41.311020 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1206/2448] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:51.305862 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=28 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:54.941485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:54.951520 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d57c8a-e633-4cdd-ad2f-da267209690b service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:54.955968 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:54.956579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:54.956867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:54.957147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:54.958940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:54.970257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:54.971740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:54.981076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1243/2449] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:04.976790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:30.682985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:30.684745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:30.703203 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f9a83-82f2-4211-ace3-c31ef784ea63 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:30.710817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:30.711473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:30.711766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:30.712056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:30.713830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:30.773597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:30.775254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:30.784375 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1207/2450] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:36.425035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:36.437483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b451fb-99ca-43c8-981b-8dfa577256f7 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:36.442937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:36.443603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:36.443886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:36.444185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:36.446477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:36.458519 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:36.460802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:36.471582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1244/2451] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:40.780070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=28 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:46.468240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:14.088928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:14.090608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:14.250733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed87cc-3e54-4004-8fde-95661b663ff5 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:14.257331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:14.257972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:14.258288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:14.258582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:14.260528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:14.272741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:14.274864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:14.285649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1208/2452] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:18.547947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:18.615777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b805c6-3fba-4b7b-aeb4-d364c1700293 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:18.621373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:18.621990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:18.622275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:18.622556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:18.624360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:18.682363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:18.683988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:18.692871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1245/2453] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:24.281122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=63 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:28.688774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=52 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:30:33.995666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:34.017585 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c6119-9ac1-498c-ba0d-a7dbebcead5d service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:34.021606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:34.022250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:34.022541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:34.022820 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:34.042073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:34.043992 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:34.054639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1209/2454] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:30:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:30:44.049618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:08.332211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:08.368451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ded08-a06c-47a6-9779-a8ed16b1b346 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:08.374026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:08.374828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:08.375119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:08.375431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:08.387732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:08.389766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:08.404444 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1246/2455] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:09.608237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:09.621959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397988e-2e64-4d7b-aea4-3681bf5efdfd service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:09.625811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:09.626439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:09.626726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:09.627044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:09.638456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:09.639938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:09.652692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1210/2456] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:18.397914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=10 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:19.516966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:19.529119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495a365-ca2d-422b-b591-82dc0f36ae48 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:19.535209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:19.535486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:19.535771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:19.536050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:19.545615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:19.547254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:19.557359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1247/2457] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:31:19.647678 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:29.552724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:08.687705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:08.698598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025cae-5c7a-4a8e-b8c5-6157c6837d53 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:08.702838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:08.703489 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:08.703781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:08.704060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:08.711827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:08.713270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:08.722771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1211/2458] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:32:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:32:18.719113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:34.376074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:34.385947 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5775-3b80-44bc-86a0-d3e351083da0 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:34.389573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa84c52-c9e5-4c26-9671-e1cfa0aeab1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:34.390180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84c52-c9e5-4c26-9671-e1cfa0aeab1a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:34.390464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84c52-c9e5-4c26-9671-e1cfa0aeab1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:34.390742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84c52-c9e5-4c26-9671-e1cfa0aeab1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:34.400008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa84c52-c9e5-4c26-9671-e1cfa0aeab1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:34.401367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa84c52-c9e5-4c26-9671-e1cfa0aeab1a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:34.431298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1248/2459] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:32:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:32:34.672665 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:34.767072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc728ad8-bb1b-40c2-8a5c-121895c81e1d service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:34.771875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:34.772518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:34.772808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:34.773086 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] PATH_INFO: `/v3/users/e7f1b0932111477caa21cd18270cd8c5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:34.774273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f1b0932111477caa21cd18270cd8c5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:34.776268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:34.971886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f1b0932111477caa21cd18270cd8c5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:34.972653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f1b0932111477caa21cd18270cd8c5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:34.981346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f1b0932111477caa21cd18270cd8c5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.144468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f1b0932111477caa21cd18270cd8c5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.157840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f1b0932111477caa21cd18270cd8c5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.164528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1212/2460] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:32:34 2022] DELETE /identity/v3/users/e7f1b0932111477caa21cd18270cd8c5 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:35.169759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:35.170148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:35.170463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:35.217476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:32:35.218783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:35.293855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1249/2461] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:32:35.304278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:35.305798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:35.382724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896b317-9884-4734-95ca-25e41f0303b9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:35.386662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1de7fe-4a76-4fdf-a2fb-76e1053b127c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:35.387324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de7fe-4a76-4fdf-a2fb-76e1053b127c service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:35.387614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de7fe-4a76-4fdf-a2fb-76e1053b127c service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:35.387895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de7fe-4a76-4fdf-a2fb-76e1053b127c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:35.394374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:35.398101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1de7fe-4a76-4fdf-a2fb-76e1053b127c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:35.399533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1de7fe-4a76-4fdf-a2fb-76e1053b127c service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:35.410207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1213/2462] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:32:35.684501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:35.686006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:35.697985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24803d95-ab8a-4d82-b4e3-ec7ea0c4c687 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:35.703392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:35.704075 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:35.704362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:35.704641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] PATH_INFO: `/v3/projects/c8fc70abe3344813a5c0b831b68f7c71` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:35.705841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8fc70abe3344813a5c0b831b68f7c71)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:35.707817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:35.870294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8fc70abe3344813a5c0b831b68f7c71', 'request_id': 'req-f201850f-80fe-4788-945f-602acdd5ba9d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.972995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8fc70abe3344813a5c0b831b68f7c71', 'request_id': 'req-f201850f-80fe-4788-945f-602acdd5ba9d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.979465 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8fc70abe3344813a5c0b831b68f7c71', 'request_id': 'req-f201850f-80fe-4788-945f-602acdd5ba9d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.991116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] The token cache is being invalidate because project c8fc70abe3344813a5c0b831b68f7c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:32:35.991440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.992315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f201850f-80fe-4788-945f-602acdd5ba9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:32:35.993879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1250/2463] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:32:35 2022] DELETE /identity/v3/projects/c8fc70abe3344813a5c0b831b68f7c71 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:36.000328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f25e8-b840-4cf7-b3a6-c15d604bd5ce None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.000844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f25e8-b840-4cf7-b3a6-c15d604bd5ce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.000951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f25e8-b840-4cf7-b3a6-c15d604bd5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.051428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d33f25e8-b840-4cf7-b3a6-c15d604bd5ce None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:32:36.052664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d33f25e8-b840-4cf7-b3a6-c15d604bd5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:36.112927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1214/2464] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:32:36.122388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98823293-c0a8-41b4-a411-b41e95629693 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.122680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98823293-c0a8-41b4-a411-b41e95629693 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.122989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98823293-c0a8-41b4-a411-b41e95629693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.143438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-98823293-c0a8-41b4-a411-b41e95629693 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:32:36.144649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-98823293-c0a8-41b4-a411-b41e95629693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:36.153599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1251/2465] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:32:36.158044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f25e8-b840-4cf7-b3a6-c15d604bd5ce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:36.167927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f25e8-b840-4cf7-b3a6-c15d604bd5ce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:36.172522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:36.173140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.173426 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.173704 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.174400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:32:36.174710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:36.176356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:36.188482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1215/2466] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:32:36.193040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98823293-c0a8-41b4-a411-b41e95629693 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:36.203602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98823293-c0a8-41b4-a411-b41e95629693 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:36.208293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:36.208920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.209194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.209478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.210194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:36.212147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:36.334020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:36.340845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1252/2467] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:32:36 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:32:36.345028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:36.356943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b075f-eebd-4b81-8539-266e0554fbc5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:36.417596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:36.418236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.418817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.419102 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.419799 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:36.421768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:36.554304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1216/2468] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:32:36 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:32:36.559299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:36.571665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050694b-0abe-4d87-b76d-2eaf5c552ba4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:36.576468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:36.577125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.577413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.577695 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.578434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:36.580096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:36.590734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1253/2469] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:32:36.594742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:36.606041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924982c0-2c21-41b5-90c7-cf3451cce79f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:36.611985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97674a1-447b-48d9-8007-f08447df1453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:36.612619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97674a1-447b-48d9-8007-f08447df1453 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.612940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97674a1-447b-48d9-8007-f08447df1453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.613218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97674a1-447b-48d9-8007-f08447df1453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.613929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97674a1-447b-48d9-8007-f08447df1453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:36.615612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97674a1-447b-48d9-8007-f08447df1453 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:36.627578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1217/2470] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:32:36.631953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:36.642370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5470142-20d1-4d9f-9330-e9950426e483 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:36.646913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:36.647619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.647906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.648187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] PATH_INFO: `/v3/projects/e51215bdae404aa6bb5518d13c1fa98b/users/c1903928948540839a7104001f844e1d/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.716646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51215bdae404aa6bb5518d13c1fa98b, user_id=c1903928948540839a7104001f844e1d, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:36.719514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:36.819676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1254/2471] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:32:36 2022] PUT /identity/v3/projects/e51215bdae404aa6bb5518d13c1fa98b/users/c1903928948540839a7104001f844e1d/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:36.825012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c615ae8-1d79-4b5a-b626-03d8996447bc None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.825298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c615ae8-1d79-4b5a-b626-03d8996447bc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.825577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c615ae8-1d79-4b5a-b626-03d8996447bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.844997 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5c615ae8-1d79-4b5a-b626-03d8996447bc None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:32:36.846130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5c615ae8-1d79-4b5a-b626-03d8996447bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:36.879167 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:36.908251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1218/2472] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:32:36.918869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:36.920645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:36.979281 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96991e20-f6e2-4336-beb5-93755018dd08 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:36.983707 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:36.984343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:36.984634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:36.984913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:36.994637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:36.996806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:37.006616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1255/2473] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:32:42.423169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05da6be-8050-42d8-ae69-4f99a4fea795 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:42.423781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05da6be-8050-42d8-ae69-4f99a4fea795 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:42.423781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05da6be-8050-42d8-ae69-4f99a4fea795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:42.460510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f05da6be-8050-42d8-ae69-4f99a4fea795 None None] MFA Rules not processed for user `c1903928948540839a7104001f844e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:32:42.461761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f05da6be-8050-42d8-ae69-4f99a4fea795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:42.533559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1219/2474] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:32:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:32:42.536238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:42.546395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281676e9-7a12-49e4-8858-d022b7746d60 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:42.551271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:42.551903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:42.552189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:42.552504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:42.553335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:42.555288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:42.652581 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1256/2475] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:32:42 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:32:42.656624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05da6be-8050-42d8-ae69-4f99a4fea795 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:42.672517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05da6be-8050-42d8-ae69-4f99a4fea795 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:42.735408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:42.736037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:42.736342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:42.736606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:42.737285 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:42.739254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:42.846970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1220/2476] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:32:42 2022] POST /identity/v3/users => generated 296 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:32:42.851178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:42.862253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233488cb-d90d-4753-874e-ad6f4365461f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:42.867898 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:42.868538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:42.868820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:42.869098 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:42.870876 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:42.872058 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:42.884047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1257/2477] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:32:42.889246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:42.900211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1402e3-1dfc-4d7c-bdcf-3fae233bb418 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:42.905100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:42.905778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:42.906067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:42.906383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] PATH_INFO: `/v3/projects/1405e11274c74fe2b39ec587d3678945/users/f5136e9fe3b9479faf9360b3969160dc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:42.922472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1405e11274c74fe2b39ec587d3678945, user_id=f5136e9fe3b9479faf9360b3969160dc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:42.925239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:43.005027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1221/2478] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:32:42 2022] PUT /identity/v3/projects/1405e11274c74fe2b39ec587d3678945/users/f5136e9fe3b9479faf9360b3969160dc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:32:43.009145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:43.020387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fef5b-c795-442c-944b-43113ee3a724 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:43.078696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88aaf2-e768-4472-abb1-1798eb41daf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:43.079373 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88aaf2-e768-4472-abb1-1798eb41daf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:43.079655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88aaf2-e768-4472-abb1-1798eb41daf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:43.079930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88aaf2-e768-4472-abb1-1798eb41daf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:43.080629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88aaf2-e768-4472-abb1-1798eb41daf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:43.082225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88aaf2-e768-4472-abb1-1798eb41daf6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:43.093273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1258/2479] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:32:43.097437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:32:43.108911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daadc7-dafc-470a-a8fa-ce53f262a73a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:32:43.113726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:32:43.114414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:43.114721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:43.115001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] PATH_INFO: `/v3/projects/1405e11274c74fe2b39ec587d3678945/users/f5136e9fe3b9479faf9360b3969160dc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:43.117019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1405e11274c74fe2b39ec587d3678945, user_id=f5136e9fe3b9479faf9360b3969160dc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:32:43.119778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:32:43.132466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1222/2480] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:32:43 2022] PUT /identity/v3/projects/1405e11274c74fe2b39ec587d3678945/users/f5136e9fe3b9479faf9360b3969160dc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:32:48.950012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c0ea0-effd-4075-bda8-c06ea4027fb5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:32:48.950943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c0ea0-effd-4075-bda8-c06ea4027fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:32:48.951173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c0ea0-effd-4075-bda8-c06ea4027fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:32:48.987608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-311c0ea0-effd-4075-bda8-c06ea4027fb5 None None] MFA Rules not processed for user `f5136e9fe3b9479faf9360b3969160dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:32:48.988804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-311c0ea0-effd-4075-bda8-c06ea4027fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:32:49.015095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:49.077750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1259/2481] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:32:53.128396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:59.083858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=20 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:11.652435 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:11.654305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:11.728222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a069b4d-67af-42f9-a9ec-ac4a128f9576 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:11.732254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:11.733074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:11.733522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:11.733961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:11.736111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:11.817888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:11.819302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:11.838926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1223/2482] 10.222.0.49 () {60 vars in 1396 bytes} [Tue Aug 2 10:33:11 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:12.028178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c0ea0-effd-4075-bda8-c06ea4027fb5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.039375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c0ea0-effd-4075-bda8-c06ea4027fb5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.045645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.046300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.046590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.046938 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.049277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:12.137827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:12.139365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:12.179046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.180585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.210103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1260/2483] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:12.250243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:12.257590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df4c9-eb5c-4d40-bb27-cb128d9abe1d service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:12.266242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:12.266979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:12.267393 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:12.267727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:12.328597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:12.330173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:12.340215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1224/2484] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:22.205233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=41 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:22.335311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=27 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:18.018440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:18.019913 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:18.093548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e136-b47b-4654-9eec-9512c1526e59 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:18.098637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:18.099305 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:18.099589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:18.099871 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:18.101704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:18.113737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:18.115370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:18.125709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1261/2485] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:34:28.121365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=32 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:34.870227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.871700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:34.885837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75975c9d-852a-42fc-b9f5-dc12b764e577 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:34.889569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:34.890211 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.890492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.890766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.892583 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:34.978474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:34.979962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:34.998335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1225/2486] 10.222.0.49 () {60 vars in 1396 bytes} [Tue Aug 2 10:34:34 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:36.047307 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:36.057642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8ea98-cf09-49a7-ba58-1e825949b24a service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:36.061338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:36.061943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:36.062222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:36.062494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:36.064253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:36.152870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:36.154329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:36.173213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1262/2487] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:44.995645 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=41 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.170592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=41 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.658637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.660626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:14.680311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301bbb4-4559-4288-8f6c-a035938a2255 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.692055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.693259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.694017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.694733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.698576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:14.713349 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.715228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.727108 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1226/2488] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:35:17.542377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:17.544674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:17.560497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d927da-0ad7-4b47-af8e-5f8bf28edf3b service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:17.565804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:17.566443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.566733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.567013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.569069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:17.584020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:17.586480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:17.611995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1263/2489] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:35:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:35:21.164725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:21.238804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d1f88-2f1a-44a1-8136-2ba52e804e93 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.246125 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.246710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.246999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.247311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] PATH_INFO: `/v3/users/c1903928948540839a7104001f844e1d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.248530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1903928948540839a7104001f844e1d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:21.252806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:21.446369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1903928948540839a7104001f844e1d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:21.447045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1903928948540839a7104001f844e1d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:21.454663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1903928948540839a7104001f844e1d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:21.531867 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1903928948540839a7104001f844e1d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:21.545061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1903928948540839a7104001f844e1d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:21.551289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1227/2490] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:35:21 2022] DELETE /identity/v3/users/c1903928948540839a7104001f844e1d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:21.787816 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:21.798997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296900f5-925b-4660-9987-30a5fc69d5de service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.859228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.859797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.860082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.860362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] PATH_INFO: `/v3/projects/e51215bdae404aa6bb5518d13c1fa98b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.861534 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51215bdae404aa6bb5518d13c1fa98b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:21.863469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:21.959433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51215bdae404aa6bb5518d13c1fa98b', 'request_id': 'req-6ed0ba28-3f70-418f-99ea-002c4b854de6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.014262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51215bdae404aa6bb5518d13c1fa98b', 'request_id': 'req-6ed0ba28-3f70-418f-99ea-002c4b854de6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.020463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51215bdae404aa6bb5518d13c1fa98b', 'request_id': 'req-6ed0ba28-3f70-418f-99ea-002c4b854de6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.033024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] The token cache is being invalidate because project e51215bdae404aa6bb5518d13c1fa98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:35:22.033440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.034109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.036061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1264/2491] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:35:21 2022] DELETE /identity/v3/projects/e51215bdae404aa6bb5518d13c1fa98b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:22.040074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:22.041695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:22.054929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc7878-d2e8-4332-a007-696629e6adea admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:22.113155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:22.113733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:22.114022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:22.114302 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] PATH_INFO: `/v3/users/f5136e9fe3b9479faf9360b3969160dc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:22.115527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5136e9fe3b9479faf9360b3969160dc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:22.117494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:22.122830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:22.268306 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5136e9fe3b9479faf9360b3969160dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.269006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5136e9fe3b9479faf9360b3969160dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.276480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5136e9fe3b9479faf9360b3969160dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.348716 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5136e9fe3b9479faf9360b3969160dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.362099 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5136e9fe3b9479faf9360b3969160dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.368120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1228/2492] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:35:22 2022] DELETE /identity/v3/users/f5136e9fe3b9479faf9360b3969160dc => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:22.610284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:22.611835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:22.630244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0ba28-3f70-418f-99ea-002c4b854de6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:22.688639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:22.689275 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:22.689557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:22.689835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] PATH_INFO: `/v3/projects/1405e11274c74fe2b39ec587d3678945` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:22.691003 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1405e11274c74fe2b39ec587d3678945)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:22.692945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:22.714370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:22.787722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1405e11274c74fe2b39ec587d3678945', 'request_id': 'req-8a8d27bb-067c-41ad-b619-6757a19b129a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.863115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1405e11274c74fe2b39ec587d3678945', 'request_id': 'req-8a8d27bb-067c-41ad-b619-6757a19b129a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.869433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1405e11274c74fe2b39ec587d3678945', 'request_id': 'req-8a8d27bb-067c-41ad-b619-6757a19b129a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.882434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] The token cache is being invalidate because project 1405e11274c74fe2b39ec587d3678945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:35:22.882831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.883531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:35:22.885171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1265/2493] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:35:22 2022] DELETE /identity/v3/projects/1405e11274c74fe2b39ec587d3678945 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:35:32.364599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:32.881188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:09.500219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:09.501768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:09.563383 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c15e0-613e-47af-b493-33866c365bea admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:09.611613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:09.612288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:09.612719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:09.613027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:09.615490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:09.685055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:09.686506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:09.697809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1229/2494] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:36:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:36:19.692535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=74 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:36.209546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.211016 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:36.269927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d27bb-067c-41ad-b619-6757a19b129a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.273546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.274168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.274453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.274732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.276539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:36.291320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.292764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:36.304227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1266/2495] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:36:36.352225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.361719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ed3c8-084d-4dd5-8e6e-3aeabcd4838b service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.366325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.366949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.367390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.367675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.369439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:36.450734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.452149 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:36.471374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1230/2496] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:36:46.298819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=26 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:46.468647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=41 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:14.628756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:14.630164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:14.641730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9bb45-fb1b-4a81-a8f5-5fb805f589cd service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:14.646120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:14.646850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:14.647351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:14.647637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:14.649391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:14.659635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:14.661143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:14.670122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1267/2497] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:37:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:37:16.524897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:16.526686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:16.543386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263131d2-e24a-4769-aeb4-85e646298d89 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:16.549922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:16.550374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:16.550865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:16.551624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:16.553523 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:16.567317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:16.569083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:16.582511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1231/2498] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:37:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:37:24.665932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:26.577632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:34.089862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:34.103602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2403ff-ecf6-4940-ade9-76d9f296f9ab service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:34.109481 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:34.110127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:34.110414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:34.110693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:34.112774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:34.235060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:34.238349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:34.257391 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1268/2499] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:37:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:37:44.248102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=33 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:38:11.958118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:38:11.974278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac166e-f020-4306-a581-8de3a27031f2 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:38:11.982543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:38:11.983268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:38:11.983760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:38:11.984132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:38:11.986635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:38:12.073640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:38:12.075942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:38:12.088047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1232/2500] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:38:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:38:22.082595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=28 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:13.799981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:13.801640 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:13.817059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c1e36-89e9-447e-85a2-8a59a8c5452c service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:13.823516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:13.824169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:13.824452 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:13.824730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:13.827237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:13.898282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:13.899886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:13.911497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1269/2501] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:39:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:39:23.906179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=28 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:23.975793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:23.977505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:24.136503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae0060-1df8-4a48-becc-a476e9c99398 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:24.141938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:24.142581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:24.142863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:24.143150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:24.145015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:24.157946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:24.159698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:24.171724 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1233/2502] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:39:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:39:34.166222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=63 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:41:13.739469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:41:13.741486 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:41:13.796897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27ecf-f5a1-4623-a9cb-4153dbc56b86 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:41:13.800746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:41:13.801372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:41:13.801656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:41:13.801941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:41:13.823507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:41:13.825336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:41:13.855720 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1270/2503] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:41:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:41:13.922914 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:41:13.942493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d1c0-d48b-49f1-938f-de725a91901a service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:41:13.951159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:41:13.952074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:41:13.952467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:41:13.952859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:41:13.955603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:41:13.962387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:41:13.973083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ef3cf-af19-4bbc-bd32-9bb4ab85119b service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:41:13.977065 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:41:13.977607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:41:13.977883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:41:13.978153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:41:13.980245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:41:14.059450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:41:14.060955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:41:14.073271 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:41:14.075606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1271/2504] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:41:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:41:14.076533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:41:14.091786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1234/2505] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:41:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:41:24.067157 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=54 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:41:24.085736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=28 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:42:18.698017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:42:18.721742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a65b8f-2d17-476b-9a14-563cfde94866 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:42:18.725828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:42:18.726483 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:42:18.726832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:42:18.727108 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:42:18.736024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:42:18.737528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:42:18.748092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1272/2506] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:42:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:42:28.743353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=10 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:14.231215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:14.242060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b045b-95af-42fb-8ab3-3706f2e74234 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:14.246339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:14.246985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:14.247304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:14.247582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:14.261338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:14.262827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:14.277608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1235/2507] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:14.286062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:14.303221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be0ac0-24b0-4903-b23b-a1b365d3deaf service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:14.310242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:14.311381 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:14.315698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:14.315698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:14.329937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:14.332845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:14.390219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1273/2508] 10.222.0.49 () {60 vars in 1396 bytes} [Tue Aug 2 10:43:14 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:14.565353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:14.581669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd58869-7e3c-4885-9794-29cbd9ab95de service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:14.592791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:14.592791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:14.593119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:14.593521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:14.609966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:14.612043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:14.625157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1236/2509] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:14.769948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:14.783592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b59b4c-c5c6-4af4-bab0-44f4c158ab61 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:14.789811 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:14.790455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:14.790740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:14.791015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:14.800573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:14.802546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:14.835394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1274/2510] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:24.619538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=12 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:24.832293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=26 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:28.048536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:28.050126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:28.067744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c112cf-b411-404e-8fdf-8f2435837ab5 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:28.073214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:28.073858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:28.074145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:28.074431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:28.084866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:28.086499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:28.097660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1237/2511] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:38.093096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:14.879700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:14.892199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b8b1-28dc-47da-a70a-3f180af96c0f service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:14.898455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:14.899096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:14.899428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:14.899734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:14.909750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:14.911694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:14.946944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1275/2512] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:45:19.585497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:19.607836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0a5c-f09d-4110-a6d8-7c01d8a59b6f service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:19.613055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd793f-dfde-473a-9dfa-a67ce7c7efd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:19.614077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd793f-dfde-473a-9dfa-a67ce7c7efd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:19.614542 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd793f-dfde-473a-9dfa-a67ce7c7efd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:19.615139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd793f-dfde-473a-9dfa-a67ce7c7efd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:19.617436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8abd793f-dfde-473a-9dfa-a67ce7c7efd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:19.781636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd793f-dfde-473a-9dfa-a67ce7c7efd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:19.783402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd793f-dfde-473a-9dfa-a67ce7c7efd8 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:19.822944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1238/2513] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:45:21.814699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:21.831999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e46bdb-4064-4eac-a65d-2fd0ac39ffe5 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:21.839533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:21.840594 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:21.840875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:21.841385 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:21.854440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:21.857080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:21.872259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1276/2514] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:45:25.436018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007f79b-ffb1-4726-9037-899a96da50cc None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:25.436909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007f79b-ffb1-4726-9037-899a96da50cc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:25.436909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007f79b-ffb1-4726-9037-899a96da50cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:25.484549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8007f79b-ffb1-4726-9037-899a96da50cc None None] MFA Rules not processed for user `d2ee2b6dd6f647179d2af999b3e62788`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:45:25.486838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8007f79b-ffb1-4726-9037-899a96da50cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:25.502776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1239/2515] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:45:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:45:25.946654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:25.948366 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:25.999310 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003eba8-2ecb-4d4c-b409-116d61e54478 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:26.011109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:26.012776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:26.013464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:26.014394 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:26.019243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:26.032365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:26.034007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:26.045761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1277/2516] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:45:26.478847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007f79b-ffb1-4726-9037-899a96da50cc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:26.514559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007f79b-ffb1-4726-9037-899a96da50cc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:26.545069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:26.545522 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:26.545749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:26.546003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:26.546288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:26.549282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:26.560066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d002e-82b6-4df5-901d-8692d74cdf94 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:26.565991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:26.567703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:26.568047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:26.568685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:26.572151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:26.575577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:26.578894 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:26.596964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1240/2517] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:45:26.597329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:26.599009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:26.619857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1278/2518] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:45:32.267133 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:32.270222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:32.284565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f8726-b5b6-48a6-a897-9cb9c9ead48d service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:32.290260 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:32.291275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:32.291772 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:32.292163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] PATH_INFO: `/v3/users/8d42f70ad3cb4c9388c09a70cfc950dc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:32.311895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d42f70ad3cb4c9388c09a70cfc950dc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:32.315076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:32.682676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d42f70ad3cb4c9388c09a70cfc950dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:32.683412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d42f70ad3cb4c9388c09a70cfc950dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:32.692493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d42f70ad3cb4c9388c09a70cfc950dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:32.886680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d42f70ad3cb4c9388c09a70cfc950dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:32.906597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d42f70ad3cb4c9388c09a70cfc950dc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:32.909714 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:32.916373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1241/2519] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:45:32 2022] DELETE /identity/v3/users/8d42f70ad3cb4c9388c09a70cfc950dc => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:45:33.462293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:33.481848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1276e-3d6d-414b-a503-4430402c6e08 service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:33.572085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:33.577243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:33.577243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:33.577243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] PATH_INFO: `/v3/projects/64f73de1986f406182cc6a52b8376b93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:33.588694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f73de1986f406182cc6a52b8376b93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:33.592212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:33.766637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f73de1986f406182cc6a52b8376b93', 'request_id': 'req-98614573-ead3-4b74-9973-a40e6e7b0f0b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:33.862206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f73de1986f406182cc6a52b8376b93', 'request_id': 'req-98614573-ead3-4b74-9973-a40e6e7b0f0b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:33.868895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f73de1986f406182cc6a52b8376b93', 'request_id': 'req-98614573-ead3-4b74-9973-a40e6e7b0f0b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:33.883090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] The token cache is being invalidate because project 64f73de1986f406182cc6a52b8376b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:45:33.883489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:33.884188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:33.885788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1279/2520] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:45:33 2022] DELETE /identity/v3/projects/64f73de1986f406182cc6a52b8376b93 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:45:33.890361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:33.892019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:33.907912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a391cb8-62f9-43c9-a282-c1a7fc13bed1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:33.974989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:33.975825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:33.976219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:33.976599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] PATH_INFO: `/v3/users/c564373bad3344be9c8d58d28276681d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:33.993198 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c564373bad3344be9c8d58d28276681d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:33.995238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:34.191520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c564373bad3344be9c8d58d28276681d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.192106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c564373bad3344be9c8d58d28276681d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.199666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c564373bad3344be9c8d58d28276681d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.273305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c564373bad3344be9c8d58d28276681d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.287300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c564373bad3344be9c8d58d28276681d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.293785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1242/2521] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:45:33 2022] DELETE /identity/v3/users/c564373bad3344be9c8d58d28276681d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:45:34.595349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:34.598428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:34.606599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:34.612400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98614573-ead3-4b74-9973-a40e6e7b0f0b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:34.687218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:34.688020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:34.688340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:34.688718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] PATH_INFO: `/v3/projects/340a38fa81ce46b6be226b1f7a2f8a3b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:34.705880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=340a38fa81ce46b6be226b1f7a2f8a3b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:34.705880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:34.853339 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340a38fa81ce46b6be226b1f7a2f8a3b', 'request_id': 'req-27e57d22-f439-476a-8c32-5ec116f679fe'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.937004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340a38fa81ce46b6be226b1f7a2f8a3b', 'request_id': 'req-27e57d22-f439-476a-8c32-5ec116f679fe'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.944851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340a38fa81ce46b6be226b1f7a2f8a3b', 'request_id': 'req-27e57d22-f439-476a-8c32-5ec116f679fe'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.958154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] The token cache is being invalidate because project 340a38fa81ce46b6be226b1f7a2f8a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:45:34.958401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.959004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:34.961295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1280/2522] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:45:34 2022] DELETE /identity/v3/projects/340a38fa81ce46b6be226b1f7a2f8a3b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:45:40.484017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:40.485702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:40.582940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:40.605017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef8c2a-9be2-422c-9369-81583866d5b4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:40.676857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:40.677560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:40.677850 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:40.678128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:40.680078 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:40.693475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:40.695142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:40.708690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1243/2523] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:45:44.954401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:51.038403 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=36 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:14.437930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:14.439453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:14.501767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d22-f439-476a-8c32-5ec116f679fe admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:14.510015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:14.510956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:14.511543 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:14.513549 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:14.514829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:14.531558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:14.533069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:14.546451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1281/2524] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:46:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:14.684718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:14.686296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:14.701100 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15309f-c5ba-4146-99ce-865c848a6136 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:14.707964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:14.708610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:14.708940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:14.709221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:14.711153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:14.798300 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:14.799973 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:14.820210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1244/2525] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:46:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:46:24.540778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=26 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:24.817309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=41 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:20.636227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:20.637696 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:20.654490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a5f4d-115d-464b-8227-44bc57720d5c service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:20.658296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:20.658917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:20.659227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:20.659513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:20.661327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:20.671976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:20.673425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:20.682529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1282/2526] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:47:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:30.678304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:14.307734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:14.318126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb693a-8cd6-4ff0-9d5a-2994722e973b service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:14.321957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:14.322521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:14.322835 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:14.323110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:14.325632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:14.340514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:14.341998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:14.353336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1245/2527] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:48:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:15.304341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:15.319216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404d16c-24d5-46f2-9430-c62fe09751b6 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:15.325048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:15.325700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:15.325984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:15.326261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:15.328147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:15.415834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:15.417420 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:15.437463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1283/2528] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:24.348351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:25.433998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=41 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:37.385070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:37.386608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:37.460173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016391fb-34af-4212-9bec-1101c91cacf1 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:37.463907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:37.467140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:37.467140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:37.467140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:37.471251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:37.560459 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:37.561949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:37.571395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1246/2529] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:48:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:37.622227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:37.631830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22636e7b-44f9-4ff1-be60-bc7995e783d0 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:37.635704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:37.636322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:37.636610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:37.636890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:37.639207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] Could not find project: 64f73de1986f406182cc6a52b8376b93.: keystone.exception.ProjectNotFound: Could not find project: 64f73de1986f406182cc6a52b8376b93. Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 02 10:48:37.666680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:48:37.669066 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:48:37.670503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:48:37.671980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:48:37.673342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 64f73de1986f406182cc6a52b8376b93. Aug 02 10:48:37.673342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Aug 02 10:48:37.673342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1284/2530] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:48:37 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 02 10:48:37.751035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:37.760307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8f035-a21d-4a07-b41b-2f0453d34f68 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:37.764819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:37.765539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:37.765831 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:37.766107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:37.768074 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:37.850340 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:37.851858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:37.862181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1247/2531] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:48:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:47.642935 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:47.857508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:32.020560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:32.022902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:32.191809 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298bc59-70f7-4a3c-9a52-dc6cca89358f service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:32.202193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:32.202894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:32.203179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:32.203501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:32.205415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:32.219587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:32.221243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:32.233355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1285/2532] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:49:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:42.227961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=54 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:13.645916 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:13.656012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b67cb-684e-4485-a764-f2cdbfaafcb7 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:13.659766 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:13.660377 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:13.660664 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:13.660941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:13.671179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:13.672637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:13.684464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1248/2533] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:14.816657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:14.857225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8f0ae-6037-4873-a4a7-0aa058c5ce24 service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:14.862984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:14.863631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:14.863910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:14.864185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:14.872172 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:14.873762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:14.883237 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1286/2534] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:15.934492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:15.935937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:15.979401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c46f94-e412-41d5-b030-bc59b8ba1a58 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:15.983092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b4479-f494-4ad3-b33f-1087eff28c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:15.983733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b4479-f494-4ad3-b33f-1087eff28c5a service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:15.984014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b4479-f494-4ad3-b33f-1087eff28c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:15.984291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b4479-f494-4ad3-b33f-1087eff28c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:15.986029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aa7b4479-f494-4ad3-b33f-1087eff28c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:16.064316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b4479-f494-4ad3-b33f-1087eff28c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:16.065771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b4479-f494-4ad3-b33f-1087eff28c5a service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:16.075243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1249/2535] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:17.602697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:17.604304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:17.619429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4add070-f360-4a9c-bd89-176167e8dd85 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:17.625040 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:17.625648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:17.625930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:17.626207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:17.627994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:17.641768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:17.643424 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:17.657842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1287/2536] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:21.072963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a1318-10b3-4a1e-9176-6ce42db6bbc1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:21.073507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a1318-10b3-4a1e-9176-6ce42db6bbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:21.073771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a1318-10b3-4a1e-9176-6ce42db6bbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:21.139366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dd0a1318-10b3-4a1e-9176-6ce42db6bbc1 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:51:21.140591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dd0a1318-10b3-4a1e-9176-6ce42db6bbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:21.206286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1250/2537] 10.222.0.49 () {58 vars in 967 bytes} [Tue Aug 2 10:51:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:51:21.359997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:21.361395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:21.452789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5694e-a779-46ea-8100-a6205144461c service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:21.457785 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a54b3c-4d44-44e6-ae74-328132969b9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:21.458397 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a54b3c-4d44-44e6-ae74-328132969b9b service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:21.458680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a54b3c-4d44-44e6-ae74-328132969b9b service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:21.458953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a54b3c-4d44-44e6-ae74-328132969b9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:21.467746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a54b3c-4d44-44e6-ae74-328132969b9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:21.469341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a54b3c-4d44-44e6-ae74-328132969b9b service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:21.477457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1288/2538] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 10:51:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:51:22.800166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a1318-10b3-4a1e-9176-6ce42db6bbc1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:22.810701 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a1318-10b3-4a1e-9176-6ce42db6bbc1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:22.814455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:22.815098 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:22.815432 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:22.815713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:22.817522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:22.824790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:22.883247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:22.884713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:22.895483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1251/2539] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:25.853654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0427b1b-c957-4204-aae1-7c27afd9eb14 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:25.854134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0427b1b-c957-4204-aae1-7c27afd9eb14 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:25.854266 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0427b1b-c957-4204-aae1-7c27afd9eb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:25.890272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f0427b1b-c957-4204-aae1-7c27afd9eb14 None None] MFA Rules not processed for user `f6880f2f7ad749beb68cd6445f3cc6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:51:25.892302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f0427b1b-c957-4204-aae1-7c27afd9eb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:25.901814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1289/2540] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:51:26.390421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:26.399515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073ae7-819e-46ee-a90b-431d055b8c7a service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:26.403745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:26.404356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:26.404652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:26.404976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:26.408084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:26.419639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:26.422896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:26.430352 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1252/2541] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:27.054104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0427b1b-c957-4204-aae1-7c27afd9eb14 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:27.068507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0427b1b-c957-4204-aae1-7c27afd9eb14 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:27.074908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:27.075774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:27.076173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:27.076829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:27.079412 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:27.093166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:27.094952 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:27.106698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1290/2542] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:36.152476 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:36.165744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36fec0-42af-42f5-81db-23ad7838f8f2 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:36.171526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:36.172173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:36.172490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:36.172771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:36.182113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:36.184311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:36.194950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1253/2543] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:37.101560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=77 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:40.035431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:40.048959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f05ee-3f8d-411a-b251-aa77c6020b2e service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:40.055338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:40.056050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:40.056407 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:40.056773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:40.058722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:40.072145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:40.073784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:40.087174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1291/2544] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:51:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:46.190553 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=38 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:50.082471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:52:25.678759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:52:25.697572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c02b54-ec92-45f7-b0eb-d065545d56fc service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:52:25.701069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:52:25.701675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:25.701956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:25.702231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:25.709295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:52:25.710746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:52:25.718490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1254/2545] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:52:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:52:35.715075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:53:16.264758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:16.276959 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035661f-066f-4e48-a248-58f15fb91846 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:16.282528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:16.283138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:16.283457 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:16.283739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:16.292072 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:16.297443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:16.302938 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1292/2546] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:53:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:17.905924 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:17.914783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e82032-cfe2-4c60-8990-9997d63b5757 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:17.918667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:17.919161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:17.919506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:17.919788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:17.930795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:17.932291 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:17.945456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1255/2547] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:53:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:26.298646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:53:27.939445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:38.403092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:38.412835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220fdc-1d5c-4a43-9018-11d69a5d6d1a service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:38.417028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:38.417689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:38.418005 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:38.418287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:38.427127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:38.428635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:38.457737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1293/2548] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:54:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:38.509552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:38.518905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aacf3b-969a-4216-9c40-9169c9bade26 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:38.523479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac9b85-4890-4848-a72a-ddaa22096f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:38.524119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac9b85-4890-4848-a72a-ddaa22096f89 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:38.524402 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac9b85-4890-4848-a72a-ddaa22096f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:38.524696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac9b85-4890-4848-a72a-ddaa22096f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:38.534614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac9b85-4890-4848-a72a-ddaa22096f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:38.535318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac9b85-4890-4848-a72a-ddaa22096f89 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:38.563076 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1256/2549] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:54:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:40.002063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:40.013774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d352-c76b-444d-802c-3f2135240932 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:40.019294 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:40.019976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:40.020261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:40.020542 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:40.030411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:40.032048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:40.043563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1294/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:54:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:46.450826 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:46.451446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:46.451446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:46.481561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] MFA Rules not processed for user `d2ee2b6dd6f647179d2af999b3e62788`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:46.481561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:46.481561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1257/2551] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:54:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:46.795259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:46.805215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f794869-1fdb-4adc-b7cd-85e5dee2f6c8 service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:46.808753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:46.809363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:46.809642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:46.809922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:46.811723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:46.821934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:46.823447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:46.832614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1295/2552] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:54:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:56.464855 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=16 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:56.828691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=25 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:07.216075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.217879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:07.303307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffcb9d-cfe5-4a7f-88fe-18a212fed225 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:07.307091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:07.308118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.308118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.308840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.310646 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] Could not find project: e51215bdae404aa6bb5518d13c1fa98b.: keystone.exception.ProjectNotFound: Could not find project: e51215bdae404aa6bb5518d13c1fa98b. Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 02 10:56:07.324293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:56:07.325640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:56:07.327161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:56:07.328582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:56:07.329984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: e51215bdae404aa6bb5518d13c1fa98b. Aug 02 10:56:07.329984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Aug 02 10:56:07.329984 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1258/2553] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:07 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 110 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 02 10:56:15.075750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:15.129671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8b7c2-77aa-49fc-aedd-42b5a232a826 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:15.133376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:15.133996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:15.134284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:15.134606 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:15.136975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:15.171805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:15.173312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:15.194653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1296/2554] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:17.314482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=33 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:19.306605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:19.319962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8caa91-0dfd-486d-a1d1-8f8867149ae4 service glance] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:19.325757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:19.326366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:19.326654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:19.326936 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:19.328803 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:19.354890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:19.356830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:19.368697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1259/2555] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:19.441589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:19.443553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:19.459562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222af47-dd8d-4dc4-8a42-6b5c35f40bb6 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:19.465338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:19.466174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:19.466786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:19.467036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] PATH_INFO: `/v3/users/9e50546af342492b9b6f6127b0ce2c61` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:19.468622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e50546af342492b9b6f6127b0ce2c61)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:19.470790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:19.722335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e50546af342492b9b6f6127b0ce2c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:19.729458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e50546af342492b9b6f6127b0ce2c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:19.730818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e50546af342492b9b6f6127b0ce2c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:19.731464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e50546af342492b9b6f6127b0ce2c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:19.743882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2d1afdd5-fb68-4607-95c1-0bce350936f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e50546af342492b9b6f6127b0ce2c61'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:19.846195 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1297/2556] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:56:19 2022] DELETE /identity/v3/users/9e50546af342492b9b6f6127b0ce2c61 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:20.190815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:20.209576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b7880-2fbb-4a57-8c03-97b9ee222c65 service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:20.286325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:20.286985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:20.287362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:20.287635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] PATH_INFO: `/v3/projects/fe6c2dfb82174a5692a746ea51769a94` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:20.304526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6c2dfb82174a5692a746ea51769a94)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:20.306635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:20.427314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6c2dfb82174a5692a746ea51769a94', 'request_id': 'req-d9864c3e-18fe-40ee-a97c-c1afc6ade357'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:20.436549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6c2dfb82174a5692a746ea51769a94', 'request_id': 'req-d9864c3e-18fe-40ee-a97c-c1afc6ade357'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:20.542128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6c2dfb82174a5692a746ea51769a94', 'request_id': 'req-d9864c3e-18fe-40ee-a97c-c1afc6ade357'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:20.559895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] The token cache is being invalidate because project fe6c2dfb82174a5692a746ea51769a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:20.560323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:20.561415 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d9864c3e-18fe-40ee-a97c-c1afc6ade357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:20.568050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1260/2557] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:56:20 2022] DELETE /identity/v3/projects/fe6c2dfb82174a5692a746ea51769a94 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:20.569634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d70943-d56d-4195-9a8a-ca2db340b528 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:20.571710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d70943-d56d-4195-9a8a-ca2db340b528 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:20.571710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d70943-d56d-4195-9a8a-ca2db340b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:20.643251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-10d70943-d56d-4195-9a8a-ca2db340b528 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:20.644823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-10d70943-d56d-4195-9a8a-ca2db340b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:20.655529 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:20.720688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1298/2558] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:20.728837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020a5c-8fb7-4c71-853a-8248f7889fd5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:20.729431 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020a5c-8fb7-4c71-853a-8248f7889fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:20.730452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020a5c-8fb7-4c71-853a-8248f7889fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:20.752657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a3020a5c-8fb7-4c71-853a-8248f7889fd5 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:20.754030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a3020a5c-8fb7-4c71-853a-8248f7889fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:20.762773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1261/2559] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:20.767244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d70943-d56d-4195-9a8a-ca2db340b528 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:20.777718 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d70943-d56d-4195-9a8a-ca2db340b528 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:20.784298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:20.785037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:20.785530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:20.785835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:20.786714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:20.787258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:20.789721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:20.805653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1299/2560] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 10:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:20.811485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020a5c-8fb7-4c71-853a-8248f7889fd5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:20.821214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020a5c-8fb7-4c71-853a-8248f7889fd5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:20.826159 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:20.826788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:20.830353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:20.830353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:20.830353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:20.830353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:20.984320 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1262/2561] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 10:56:20 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:20.989091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:21.000949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42bb6a-1b13-4294-a82e-60f3bf4ff231 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:21.070156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:21.070685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:21.070965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:21.071277 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:21.071964 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:21.074102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:21.177060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1300/2562] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 10:56:20 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:21.181323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:21.191912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2217f-747a-44d5-a604-ab990a847389 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:21.196824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:21.198018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:21.198018 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:21.198279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:21.198879 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:21.200671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:21.212301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1263/2563] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:21.217523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:21.228656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3305e-ef1d-4944-9389-7c3777f466be admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:21.234597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de6bb4-64f4-44bf-b413-07c0162cb16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:21.237802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de6bb4-64f4-44bf-b413-07c0162cb16d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:21.237802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de6bb4-64f4-44bf-b413-07c0162cb16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:21.237802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de6bb4-64f4-44bf-b413-07c0162cb16d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:21.237802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de6bb4-64f4-44bf-b413-07c0162cb16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:21.239010 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de6bb4-64f4-44bf-b413-07c0162cb16d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:21.252116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1301/2564] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 10:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:21.256385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:21.268170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e0cf2-71ce-4ce9-9f8d-33ca63b08682 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:21.274635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:21.275366 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:21.275657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:21.275961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] PATH_INFO: `/v3/projects/886baa7c44f641d287f911a81bebaa13/users/89de93e6f9564e2cb73500e3d383e836/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:21.294907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=886baa7c44f641d287f911a81bebaa13, user_id=89de93e6f9564e2cb73500e3d383e836, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:21.297756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:21.301463 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:21.433279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1264/2565] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 10:56:21 2022] PUT /identity/v3/projects/886baa7c44f641d287f911a81bebaa13/users/89de93e6f9564e2cb73500e3d383e836/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:21.438554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:21.439097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:21.439410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:21.459556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:21.460839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:21.525403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1302/2566] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:21.536288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:21.538134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:21.626691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858479d0-5f74-468a-a26a-e1c9a16af9fb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:21.630494 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36cb0f-9056-4517-8e62-a3f34701bfd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:21.631132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36cb0f-9056-4517-8e62-a3f34701bfd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:21.631452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36cb0f-9056-4517-8e62-a3f34701bfd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:21.631824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36cb0f-9056-4517-8e62-a3f34701bfd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:21.642512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36cb0f-9056-4517-8e62-a3f34701bfd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:21.644069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36cb0f-9056-4517-8e62-a3f34701bfd7 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:21.655853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1265/2567] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:23.405888 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:23.407614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:23.425063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f5519-8b84-4be2-8dcd-05475805357d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:23.429024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:23.432846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:23.432846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:23.432846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:23.432846 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:23.458823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:23.530741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:23.535322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:23.551286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1303/2568] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:28.682654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f521e-ea62-44ae-82a7-5ed5fcc6c84e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.684528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f521e-ea62-44ae-82a7-5ed5fcc6c84e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.684528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f521e-ea62-44ae-82a7-5ed5fcc6c84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.724963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d43f521e-ea62-44ae-82a7-5ed5fcc6c84e None None] MFA Rules not processed for user `89de93e6f9564e2cb73500e3d383e836`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:28.726256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d43f521e-ea62-44ae-82a7-5ed5fcc6c84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.805033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1266/2569] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:28.820717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.822541 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.842073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597b2e-193b-4874-adcd-e457dd69b769 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.850333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.851301 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.851589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.851863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.861738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.863327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.877504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1304/2570] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:32.586656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:32.587138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:32.587476 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:32.613224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] MFA Rules not processed for user `89de93e6f9564e2cb73500e3d383e836`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:32.615153 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:32.631351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1267/2571] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:32.961340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:32.975167 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c448c2-f903-488a-a391-8949da414f6c service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:32.979345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:32.979979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:32.980309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:32.980836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:32.982734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:32.993470 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:32.994925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.013047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1305/2572] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:33.540698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:33.544949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:33.659248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d0b6-b591-4d42-b675-0cf77e62eb00 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:33.674039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:33.674685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.674972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.677974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.677974 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:33.690096 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:33.691740 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.700657 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:33.706929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1268/2573] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:43.001257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=41 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:43.693451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:43.706808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bece7-59c8-4eef-aa91-38ea2070ef48 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:43.712906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:43.713537 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:43.714018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:43.715215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:43.718120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:43.734200 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:43.735741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.747392 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1306/2574] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:43.854756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:43.867612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816dee8-5aa3-40cc-84e2-08bf971d51db service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:43.874314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:43.874959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:43.875276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:43.875577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:43.877394 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:43.893455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:43.895798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.912972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1269/2575] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:44.306039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:44.307485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:44.322628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84668333-f515-4a45-9972-e2d3e4a3a442 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:44.330344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:44.331012 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:44.331462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:44.331961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:44.334171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:44.347395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:44.349249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:44.361770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1307/2576] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:44.484397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:44.486516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:44.503882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ade4d0-1476-49ba-979c-390a5b960abf service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:44.508589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:44.509256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:44.509546 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:44.509828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] PATH_INFO: `/v3/users/1a800d95ac1d492bb44947fdf5bb3377` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:44.511259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a800d95ac1d492bb44947fdf5bb3377)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:44.513267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:44.716688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a800d95ac1d492bb44947fdf5bb3377'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:44.717337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a800d95ac1d492bb44947fdf5bb3377'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:44.724206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a800d95ac1d492bb44947fdf5bb3377'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:44.794197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a800d95ac1d492bb44947fdf5bb3377'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:44.817154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a800d95ac1d492bb44947fdf5bb3377'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:44.823679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1270/2577] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 10:56:44 2022] DELETE /identity/v3/users/1a800d95ac1d492bb44947fdf5bb3377 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:45.086875 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:45.098341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e163b-39a5-4910-8d72-2db83192480e service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:45.164015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:45.164711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:45.165011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:45.165302 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] PATH_INFO: `/v3/projects/356ad325d7914771a7e1b66caf2b6e4d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:45.166474 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=356ad325d7914771a7e1b66caf2b6e4d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:45.168893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:45.319083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356ad325d7914771a7e1b66caf2b6e4d', 'request_id': 'req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:45.406259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356ad325d7914771a7e1b66caf2b6e4d', 'request_id': 'req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:45.414328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356ad325d7914771a7e1b66caf2b6e4d', 'request_id': 'req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:45.426996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] The token cache is being invalidate because project 356ad325d7914771a7e1b66caf2b6e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:45.427328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:45.428000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:45.429619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1308/2578] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 10:56:45 2022] DELETE /identity/v3/projects/356ad325d7914771a7e1b66caf2b6e4d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:47.880765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:47.882480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:47.970516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaceefb-3910-4b5e-8d4e-adf74fa8d084 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:48.031000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:48.044582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:48.044824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:48.045060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:48.045341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:48.047237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:48.061417 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:48.063065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:48.075703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1271/2579] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:56:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:55.423676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:58.142571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=8 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:31.337723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:31.339004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:31.397902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da3b1b-ec10-4aad-ba78-b54e0f2896a5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:31.401799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:31.402427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:31.402714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:31.402995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:31.404881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:31.415009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:31.416484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:31.424713 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1309/2580] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:57:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:57:41.420980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=26 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:58:16.057395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:58:16.058963 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:16.118748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f893f87-5431-4cee-9b4d-9a0824b4b5be service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:58:16.122439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a199e47-64cf-49a9-b3f2-216dc17a9f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:58:16.123070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a199e47-64cf-49a9-b3f2-216dc17a9f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:16.123396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a199e47-64cf-49a9-b3f2-216dc17a9f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:16.123680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a199e47-64cf-49a9-b3f2-216dc17a9f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:16.125517 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8a199e47-64cf-49a9-b3f2-216dc17a9f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:16.207387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a199e47-64cf-49a9-b3f2-216dc17a9f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:58:16.208870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a199e47-64cf-49a9-b3f2-216dc17a9f3e service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:58:16.229632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1272/2581] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:58:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:58:26.225119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=61 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:58:27.065800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:58:27.067312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:27.078186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9dcb8-2fe8-46cc-a9b5-e56ed4e5b92b service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:58:27.081963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:58:27.082579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:27.082861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:27.083137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:27.085122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:27.099338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:58:27.101326 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:58:27.113009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1310/2582] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:58:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:58:37.107828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:59:21.537528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9ae7-fb90-44d9-b81a-690a44ce2b4c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:21.538082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9ae7-fb90-44d9-b81a-690a44ce2b4c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:21.538195 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9ae7-fb90-44d9-b81a-690a44ce2b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:21.559496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cade9ae7-fb90-44d9-b81a-690a44ce2b4c None None] MFA Rules not processed for user `89de93e6f9564e2cb73500e3d383e836`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:59:21.560811 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cade9ae7-fb90-44d9-b81a-690a44ce2b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:21.618370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1273/2583] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 10:59:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:59:31.611448 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=25 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:59:36.768754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:59:36.770413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:36.817367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e82f8-64c8-46e3-bdfc-230a78a70e42 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:59:36.823899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:59:36.828894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:36.828894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:36.828894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:36.828894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:36.841208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:59:36.842843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:59:36.854868 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1311/2584] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:59:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:59:37.193807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9ae7-fb90-44d9-b81a-690a44ce2b4c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:59:37.204916 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade9ae7-fb90-44d9-b81a-690a44ce2b4c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:59:37.208686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:59:37.209335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:37.209617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:37.210407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:37.212347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:37.300961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:59:37.313493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:59:37.313493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1274/2585] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:59:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:59:45.885271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:59:45.887447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:45.981451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df72cf-604f-42f4-a5e1-3daf815ec182 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:59:45.986917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:59:45.987591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:45.987881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:45.988155 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:45.989945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:46.003224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:59:46.004851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:59:46.016418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1312/2586] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 10:59:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:59:47.309064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=33 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:59:56.010884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=60 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:38.757068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:38.769710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11649096-cbc7-4eb3-adc6-6c49c303efe3 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:38.774128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:38.774761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:38.775054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:38.775363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:38.777247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:38.848534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:38.850212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:38.861443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1275/2587] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:00:43.260206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:43.262024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:43.279955 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e45f6-be18-4ed0-9095-1a373f63c25f service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:43.286110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:43.286110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:43.286110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:43.286110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] PATH_INFO: `/v3/users/f6880f2f7ad749beb68cd6445f3cc6d6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:43.287276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6880f2f7ad749beb68cd6445f3cc6d6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:43.289223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:43.512330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6880f2f7ad749beb68cd6445f3cc6d6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:43.519749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6880f2f7ad749beb68cd6445f3cc6d6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:43.520554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6880f2f7ad749beb68cd6445f3cc6d6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:43.521308 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6880f2f7ad749beb68cd6445f3cc6d6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:43.538554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bc4d855b-422c-4002-a65e-9747c700cfe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6880f2f7ad749beb68cd6445f3cc6d6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:43.629096 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1313/2588] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:00:43 2022] DELETE /identity/v3/users/f6880f2f7ad749beb68cd6445f3cc6d6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:43.935276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:43.946870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975f13a-3c96-46c8-9c81-5222d49f98f1 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.027337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.027852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.028150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.028433 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] PATH_INFO: `/v3/projects/662f508ce05e4a85896f30f772be6410` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.029673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=662f508ce05e4a85896f30f772be6410)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.031666 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:44.204752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '662f508ce05e4a85896f30f772be6410', 'request_id': 'req-d4bf61d3-a519-43a2-a867-4b6141934085'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:44.214780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '662f508ce05e4a85896f30f772be6410', 'request_id': 'req-d4bf61d3-a519-43a2-a867-4b6141934085'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:44.295660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '662f508ce05e4a85896f30f772be6410', 'request_id': 'req-d4bf61d3-a519-43a2-a867-4b6141934085'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:44.308743 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] The token cache is being invalidate because project 662f508ce05e4a85896f30f772be6410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:00:44.312551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:44.312551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d4bf61d3-a519-43a2-a867-4b6141934085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:44.312551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1276/2589] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:00:43 2022] DELETE /identity/v3/projects/662f508ce05e4a85896f30f772be6410 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:44.318311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420af64-15c9-4aed-9b14-a0985a51580d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.318857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420af64-15c9-4aed-9b14-a0985a51580d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.318857 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420af64-15c9-4aed-9b14-a0985a51580d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.340693 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6420af64-15c9-4aed-9b14-a0985a51580d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:44.341954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6420af64-15c9-4aed-9b14-a0985a51580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:44.413199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1314/2590] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:44.415107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e49b6-f5c0-4fdb-9f0a-8a0516e1df2c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.415538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e49b6-f5c0-4fdb-9f0a-8a0516e1df2c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.415824 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e49b6-f5c0-4fdb-9f0a-8a0516e1df2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.438663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-587e49b6-f5c0-4fdb-9f0a-8a0516e1df2c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:44.439910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-587e49b6-f5c0-4fdb-9f0a-8a0516e1df2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:44.449437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1277/2591] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:44.453705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420af64-15c9-4aed-9b14-a0985a51580d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:44.463931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420af64-15c9-4aed-9b14-a0985a51580d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.468608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.469234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.469516 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.469795 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.470523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:00:44.470826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.472496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:44.485607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1315/2592] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:00:44.490840 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e49b6-f5c0-4fdb-9f0a-8a0516e1df2c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:44.502720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e49b6-f5c0-4fdb-9f0a-8a0516e1df2c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.508670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.509295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.509582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.509952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.510675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.512919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:44.535934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:44.638466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1278/2593] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:44 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:44.639869 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:44.650399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bbfbb-33db-4894-83f2-6bba2af28800 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.707843 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.708467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.708753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.709032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.709738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.711736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:44.719671 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:44.841934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1316/2594] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:44 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:44.846609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:44.858141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd96cb-43d1-41fd-adb9-ed6d7aa868c6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.862960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.863625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.863908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.864192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.864893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.866508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:44.877055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1279/2595] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:44.881620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:44.891649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e69ee2-7513-4427-9612-a352bfa11cd3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.896793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922485b-264e-44d7-9125-f91bb9885df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.897402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922485b-264e-44d7-9125-f91bb9885df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.897687 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922485b-264e-44d7-9125-f91bb9885df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.897967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922485b-264e-44d7-9125-f91bb9885df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.898666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922485b-264e-44d7-9125-f91bb9885df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.900369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922485b-264e-44d7-9125-f91bb9885df2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:44.911647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1317/2596] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:44.915783 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:44.927359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d5c8-f570-4c09-ac0d-74052bf61ddd admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.933268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.937058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.937058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.937058 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] PATH_INFO: `/v3/projects/9b5d842ccfca402e824fc4bce7b8d85b/users/bcafe82d93c64891a4eff34c3fcefad7/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.951292 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5d842ccfca402e824fc4bce7b8d85b, user_id=bcafe82d93c64891a4eff34c3fcefad7, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.953968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:45.072116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1280/2597] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:44 2022] PUT /identity/v3/projects/9b5d842ccfca402e824fc4bce7b8d85b/users/bcafe82d93c64891a4eff34c3fcefad7/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:45.083545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa4068-1fae-461b-b95b-2baa7f4cbc49 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:45.083840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa4068-1fae-461b-b95b-2baa7f4cbc49 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:45.084124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa4068-1fae-461b-b95b-2baa7f4cbc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:45.111716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b4aa4068-1fae-461b-b95b-2baa7f4cbc49 None None] MFA Rules not processed for user `bcafe82d93c64891a4eff34c3fcefad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:45.112940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b4aa4068-1fae-461b-b95b-2baa7f4cbc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:45.185107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1318/2598] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:45.194391 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:45.208479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:45.278365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8f4c-bb3e-4c98-847d-2f99a2fe45c0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:45.284263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16367286-ef85-4ade-bcd5-b1a082af57b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:45.284884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16367286-ef85-4ade-bcd5-b1a082af57b9 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:45.285170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16367286-ef85-4ade-bcd5-b1a082af57b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:45.285479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16367286-ef85-4ade-bcd5-b1a082af57b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:45.293538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16367286-ef85-4ade-bcd5-b1a082af57b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:45.295141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16367286-ef85-4ade-bcd5-b1a082af57b9 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:45.310805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1281/2599] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:45.534454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa4068-1fae-461b-b95b-2baa7f4cbc49 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:45.603923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa4068-1fae-461b-b95b-2baa7f4cbc49 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:45.609225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:45.609897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:45.615378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:45.615808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] PATH_INFO: `/v3/users/bcafe82d93c64891a4eff34c3fcefad7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:45.620061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcafe82d93c64891a4eff34c3fcefad7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:45.625563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:45.885800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcafe82d93c64891a4eff34c3fcefad7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:45.895624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcafe82d93c64891a4eff34c3fcefad7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:45.896737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcafe82d93c64891a4eff34c3fcefad7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:45.897402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcafe82d93c64891a4eff34c3fcefad7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:45.909293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:45.916815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcafe82d93c64891a4eff34c3fcefad7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:46.015491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1319/2600] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:00:45 2022] DELETE /identity/v3/users/bcafe82d93c64891a4eff34c3fcefad7 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:46.020614 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:46.021114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:46.021251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:46.041313 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:46.042631 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:46.112814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1282/2601] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:46.122555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:46.124036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:46.205656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3809a-dc45-48f8-b290-bde5ed26bb9e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:46.205656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6af471-436e-4a9b-a093-f7b05181946d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:46.205656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6af471-436e-4a9b-a093-f7b05181946d service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:46.205656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6af471-436e-4a9b-a093-f7b05181946d service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:46.206243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6af471-436e-4a9b-a093-f7b05181946d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:46.206243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6af471-436e-4a9b-a093-f7b05181946d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:46.206243 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6af471-436e-4a9b-a093-f7b05181946d service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:46.214772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1320/2602] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:46.646017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:46.647505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:46.661525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb41c60-cc11-4e6e-a41e-3184bcf5b65f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:46.666778 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:46.667438 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:46.667726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:46.668006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] PATH_INFO: `/v3/projects/9b5d842ccfca402e824fc4bce7b8d85b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:46.669196 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5d842ccfca402e824fc4bce7b8d85b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:46.672573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:46.814519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:46.820717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5d842ccfca402e824fc4bce7b8d85b', 'request_id': 'req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:46.827504 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5d842ccfca402e824fc4bce7b8d85b', 'request_id': 'req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:46.889785 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5d842ccfca402e824fc4bce7b8d85b', 'request_id': 'req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:46.901987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] The token cache is being invalidate because project 9b5d842ccfca402e824fc4bce7b8d85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:00:46.902282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:46.902939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5b6bd37b-1a2c-4644-9c5e-7a158bab892a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:46.904479 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1283/2603] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:00:46 2022] DELETE /identity/v3/projects/9b5d842ccfca402e824fc4bce7b8d85b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:46.909616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be8b0-e118-43af-9175-988db9acf963 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:46.910110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be8b0-e118-43af-9175-988db9acf963 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:46.910234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be8b0-e118-43af-9175-988db9acf963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:46.929812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-730be8b0-e118-43af-9175-988db9acf963 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:46.934831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-730be8b0-e118-43af-9175-988db9acf963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:46.991528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1321/2604] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:46.999540 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deae37-63d7-492b-846b-8aca4cf8b601 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:46.999837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deae37-63d7-492b-846b-8aca4cf8b601 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.000134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deae37-63d7-492b-846b-8aca4cf8b601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.021571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-70deae37-63d7-492b-846b-8aca4cf8b601 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:47.022780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-70deae37-63d7-492b-846b-8aca4cf8b601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:47.030920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1284/2605] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:47.035813 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be8b0-e118-43af-9175-988db9acf963 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.045866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be8b0-e118-43af-9175-988db9acf963 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.050688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.051464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.051758 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.052036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.052725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:00:47.053031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:47.054725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:47.067388 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1322/2606] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:00:47.075534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70deae37-63d7-492b-846b-8aca4cf8b601 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.085524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70deae37-63d7-492b-846b-8aca4cf8b601 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.090201 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.090816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.091105 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.091418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.092090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:47.094007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:47.183334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1285/2607] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:47 2022] POST /identity/v3/projects => generated 401 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:47.187907 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.198858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abe24a-3196-49ec-96b1-114932927f63 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.262504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.263129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.263446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.263726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.264401 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:47.266364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:47.365797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1323/2608] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:47 2022] POST /identity/v3/users => generated 301 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:47.367326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.377944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4f586-4381-49d7-93d7-a197e53dfaf5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.387084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.387744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.388617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.388617 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.389000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:47.390847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:47.401972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1286/2609] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:47.407837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.418844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d24b82-bf82-43a3-aa31-9fdcce2237b5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.424163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7081b-4abc-482b-a047-2b3ef41c3ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.425077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7081b-4abc-482b-a047-2b3ef41c3ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.425431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7081b-4abc-482b-a047-2b3ef41c3ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.425776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7081b-4abc-482b-a047-2b3ef41c3ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.427091 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7081b-4abc-482b-a047-2b3ef41c3ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:47.428985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7081b-4abc-482b-a047-2b3ef41c3ea0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:47.430665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:47.446725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1324/2610] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:47.448160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.458985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b21777-3201-4b0d-a07e-2fad5145e80f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.467470 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.474903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.474903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.474903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] PATH_INFO: `/v3/projects/91b4823ab11f45d29ff4109526a028cb/users/334a5cac422a4e5da04828738f36a811/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.494961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b4823ab11f45d29ff4109526a028cb, user_id=334a5cac422a4e5da04828738f36a811, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:47.497806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:47.638971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1287/2611] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:47 2022] PUT /identity/v3/projects/91b4823ab11f45d29ff4109526a028cb/users/334a5cac422a4e5da04828738f36a811/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:47.650572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344878-b11a-4531-b8b0-4edb36300202 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.650885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344878-b11a-4531-b8b0-4edb36300202 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.651208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344878-b11a-4531-b8b0-4edb36300202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.682570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-48344878-b11a-4531-b8b0-4edb36300202 None None] MFA Rules not processed for user `334a5cac422a4e5da04828738f36a811`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:47.683841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-48344878-b11a-4531-b8b0-4edb36300202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:47.750309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1325/2612] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:47 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:47.755462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.828267 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b4ade-8317-4f7d-b0b8-e70f17a5b2bd admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.833681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.834641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.834641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:47.835169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:47.836054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:47.838034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:47.926079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1288/2613] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:47 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:47.930101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48344878-b11a-4531-b8b0-4edb36300202 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:47.941120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48344878-b11a-4531-b8b0-4edb36300202 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:47.998944 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:47.999597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:47.999948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.000189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.000851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:48.002780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:48.134312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1326/2614] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:47 2022] POST /identity/v3/users => generated 308 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:48.138199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:48.148709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2e6ba-6f36-405a-b84a-9ccae913c32c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:48.153422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:48.154041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:48.154328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.154607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.155379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:48.157004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:48.173212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1289/2615] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:48.179548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:48.188902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357078eb-4ab8-4154-b7b3-71d4e2d1aa22 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:48.194217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:48.194886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:48.195174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.195491 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] PATH_INFO: `/v3/projects/c980fb245295459fb0058c2fcede1256/users/f3b5edc286394a37b319d2a8ebbde6af/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.210263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c980fb245295459fb0058c2fcede1256, user_id=f3b5edc286394a37b319d2a8ebbde6af, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:48.213026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:48.302165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1327/2616] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:48 2022] PUT /identity/v3/projects/c980fb245295459fb0058c2fcede1256/users/f3b5edc286394a37b319d2a8ebbde6af/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:48.306445 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:48.316601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516f376-36bc-4412-b6a4-37f984f8754f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:48.356443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:48.375221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd1ba8-5c77-4da2-b615-d968760f3555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:48.379742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd1ba8-5c77-4da2-b615-d968760f3555 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:48.379742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd1ba8-5c77-4da2-b615-d968760f3555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.379742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd1ba8-5c77-4da2-b615-d968760f3555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.379742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd1ba8-5c77-4da2-b615-d968760f3555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:48.379742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd1ba8-5c77-4da2-b615-d968760f3555 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:48.388907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1290/2617] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:00:48.393290 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:48.404180 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fc5aa-242d-4b8c-876a-c382f7a6c182 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:48.409552 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:48.411208 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:48.411372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.411641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] PATH_INFO: `/v3/projects/c980fb245295459fb0058c2fcede1256/users/f3b5edc286394a37b319d2a8ebbde6af/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.413684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c980fb245295459fb0058c2fcede1256, user_id=f3b5edc286394a37b319d2a8ebbde6af, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:48.416447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:48.434185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1328/2618] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:48 2022] PUT /identity/v3/projects/c980fb245295459fb0058c2fcede1256/users/f3b5edc286394a37b319d2a8ebbde6af/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:48.441348 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd36a-66b6-4ec4-9eb6-7a6037ab79b5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:48.441741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd36a-66b6-4ec4-9eb6-7a6037ab79b5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.442103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd36a-66b6-4ec4-9eb6-7a6037ab79b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.470691 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-847fd36a-66b6-4ec4-9eb6-7a6037ab79b5 None None] MFA Rules not processed for user `f3b5edc286394a37b319d2a8ebbde6af`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:48.472083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-847fd36a-66b6-4ec4-9eb6-7a6037ab79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:48.543774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1291/2619] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:48 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:48.555455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:48.557153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:48.600917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:48.627673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb19ee-9de7-457f-8c2f-2f130ff25425 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:48.631768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9e5bf-a074-4def-9ac7-5eacacddf243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:48.632418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9e5bf-a074-4def-9ac7-5eacacddf243 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:48.632969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9e5bf-a074-4def-9ac7-5eacacddf243 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.633624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9e5bf-a074-4def-9ac7-5eacacddf243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.643018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9e5bf-a074-4def-9ac7-5eacacddf243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:48.644475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9e5bf-a074-4def-9ac7-5eacacddf243 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:48.653847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1329/2620] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:48 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:48.792718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fd36a-66b6-4ec4-9eb6-7a6037ab79b5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:48.855948 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fd36a-66b6-4ec4-9eb6-7a6037ab79b5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:48.863938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:48.864602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:48.864887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:48.865386 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] PATH_INFO: `/v3/users/334a5cac422a4e5da04828738f36a811` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:48.866582 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=334a5cac422a4e5da04828738f36a811)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:48.868597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:49.069013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334a5cac422a4e5da04828738f36a811'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:49.069636 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334a5cac422a4e5da04828738f36a811'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:49.076559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334a5cac422a4e5da04828738f36a811'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:49.147390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334a5cac422a4e5da04828738f36a811'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:49.148837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:49.161324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334a5cac422a4e5da04828738f36a811'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:49.167329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1292/2621] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:00:48 2022] DELETE /identity/v3/users/334a5cac422a4e5da04828738f36a811 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:49.172018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:49.172495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:49.172636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:49.193450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:49.194648 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:49.257334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1330/2622] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:49.267084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:49.268931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:49.327662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710dd2-099c-4f41-8d63-ba851217fa5d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:49.331332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:49.331955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:49.332890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:49.333550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:49.343329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:49.344845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:49.354975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1293/2623] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:49.846646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:49.848238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:49.862139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4312f5-e59f-4e32-b49c-2cc60aa9e857 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:49.867492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:49.868248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:49.872216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:49.872216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] PATH_INFO: `/v3/projects/91b4823ab11f45d29ff4109526a028cb` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:49.872216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b4823ab11f45d29ff4109526a028cb)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:49.872216 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:50.038560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b4823ab11f45d29ff4109526a028cb', 'request_id': 'req-f6450755-25d6-4d31-86ac-e83044e05b54'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.095895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b4823ab11f45d29ff4109526a028cb', 'request_id': 'req-f6450755-25d6-4d31-86ac-e83044e05b54'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.102942 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b4823ab11f45d29ff4109526a028cb', 'request_id': 'req-f6450755-25d6-4d31-86ac-e83044e05b54'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.115892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] The token cache is being invalidate because project 91b4823ab11f45d29ff4109526a028cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:00:50.116186 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.116849 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.118618 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1331/2624] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:00:49 2022] DELETE /identity/v3/projects/91b4823ab11f45d29ff4109526a028cb => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:50.122681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:50.132640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9408-3b7c-4769-b314-be401bb68047 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:50.192804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:50.193608 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:50.193732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:50.193994 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] PATH_INFO: `/v3/users/f3b5edc286394a37b319d2a8ebbde6af` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:50.195207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b5edc286394a37b319d2a8ebbde6af)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:50.197173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:50.442077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b5edc286394a37b319d2a8ebbde6af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.442804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b5edc286394a37b319d2a8ebbde6af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.450166 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b5edc286394a37b319d2a8ebbde6af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.555782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b5edc286394a37b319d2a8ebbde6af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.575245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-226f91a5-8605-4f98-8344-76d42c329179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b5edc286394a37b319d2a8ebbde6af'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:50.578325 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1294/2625] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:00:50 2022] DELETE /identity/v3/users/f3b5edc286394a37b319d2a8ebbde6af => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:51.239492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:51.242218 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:51.256030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6450755-25d6-4d31-86ac-e83044e05b54 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:51.314656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:51.324117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:51.324772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:51.325060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:51.325341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] PATH_INFO: `/v3/projects/c980fb245295459fb0058c2fcede1256` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:51.326510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] RBAC: Authorizing `identity:delete_project(project_id=c980fb245295459fb0058c2fcede1256)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:51.328471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:51.452688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c980fb245295459fb0058c2fcede1256', 'request_id': 'req-8b7652f4-aa59-4384-b581-9ad1fb0e9def'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:51.524466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c980fb245295459fb0058c2fcede1256', 'request_id': 'req-8b7652f4-aa59-4384-b581-9ad1fb0e9def'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:51.536149 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c980fb245295459fb0058c2fcede1256', 'request_id': 'req-8b7652f4-aa59-4384-b581-9ad1fb0e9def'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:51.548561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] The token cache is being invalidate because project c980fb245295459fb0058c2fcede1256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:00:51.548890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:51.549561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8b7652f4-aa59-4384-b581-9ad1fb0e9def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:51.551927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1332/2626] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:00:51 2022] DELETE /identity/v3/projects/c980fb245295459fb0058c2fcede1256 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:51.556909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46f1d9-8f6f-438c-8a75-98f2d220d73b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:51.557379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46f1d9-8f6f-438c-8a75-98f2d220d73b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:51.557379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46f1d9-8f6f-438c-8a75-98f2d220d73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:51.566449 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:51.588032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0d46f1d9-8f6f-438c-8a75-98f2d220d73b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:51.589284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0d46f1d9-8f6f-438c-8a75-98f2d220d73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:51.657469 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1295/2627] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:51.665992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc786-0235-421a-9b4b-500118301063 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:51.666284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc786-0235-421a-9b4b-500118301063 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:51.666567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc786-0235-421a-9b4b-500118301063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:51.688323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f07cc786-0235-421a-9b4b-500118301063 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:51.689511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f07cc786-0235-421a-9b4b-500118301063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:51.697692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1333/2628] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:51.701768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46f1d9-8f6f-438c-8a75-98f2d220d73b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:51.711761 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46f1d9-8f6f-438c-8a75-98f2d220d73b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:51.716362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:51.716982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:51.717270 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:51.717552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:51.718280 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:00:51.718597 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:51.720251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:51.732814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1296/2629] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:00:51.737100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc786-0235-421a-9b4b-500118301063 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:51.747070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc786-0235-421a-9b4b-500118301063 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:51.752463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:51.753077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:51.753355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:51.753628 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:51.754352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:51.756265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:51.839317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1334/2630] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:51 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:51.841969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:51.853786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d486-504f-47f3-83ce-850e626818d8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:51.911237 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:51.911868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:51.912150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:51.912423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:51.913414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:51.915014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:51.996379 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1297/2631] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:51 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:52.000280 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:52.011738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25909-f3a8-4648-8ffc-9b0bde718fd0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:52.016957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:52.017229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.017458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.017751 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:52.018742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:52.020633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:52.167095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1335/2632] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:52.168934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:52.187461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6be9-a203-4b0f-9e82-0508378a9006 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:52.193709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845b122-0777-4bd3-9a90-2b0b929db178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:52.194420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845b122-0777-4bd3-9a90-2b0b929db178 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.194773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845b122-0777-4bd3-9a90-2b0b929db178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.195118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845b122-0777-4bd3-9a90-2b0b929db178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:52.195995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845b122-0777-4bd3-9a90-2b0b929db178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:52.197689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845b122-0777-4bd3-9a90-2b0b929db178 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:52.214680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1298/2633] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:52.218884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:52.232764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ecce3-3a32-44a5-92f9-2799ea3702b7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:52.237556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:52.238251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.238571 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.238851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] PATH_INFO: `/v3/projects/93ad93a10d124e0b94cf0fb6f918d8ee/users/27dcd3892ed84559ae338dff5afd0cc5/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:52.255061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ad93a10d124e0b94cf0fb6f918d8ee, user_id=27dcd3892ed84559ae338dff5afd0cc5, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:52.258782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:52.343727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1336/2634] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:52 2022] PUT /identity/v3/projects/93ad93a10d124e0b94cf0fb6f918d8ee/users/27dcd3892ed84559ae338dff5afd0cc5/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:52.355451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b3f4-37a3-4902-901a-95d96c5731a4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.355753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b3f4-37a3-4902-901a-95d96c5731a4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.356015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b3f4-37a3-4902-901a-95d96c5731a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:52.386706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-80d1b3f4-37a3-4902-901a-95d96c5731a4 None None] MFA Rules not processed for user `27dcd3892ed84559ae338dff5afd0cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:52.387886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-80d1b3f4-37a3-4902-901a-95d96c5731a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:52.462107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:52.472160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1299/2635] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:52.474160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:52.549988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652565a-5502-4133-af3c-c22ec60fa80d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:52.555164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:52.555886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.556173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.556459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:52.557185 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:52.559306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:52.680614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1337/2636] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:52 2022] POST /identity/v3/projects => generated 361 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:52.684793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1b3f4-37a3-4902-901a-95d96c5731a4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:52.704711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1b3f4-37a3-4902-901a-95d96c5731a4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:52.763336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:52.763965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.764245 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.764518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:52.765181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:52.767092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:52.882881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1300/2637] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:52 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:52.886990 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:52.894818 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:52.899084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0fcbf-76c3-469b-a098-e74f7b5e212e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:52.904083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:52.904919 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.905284 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.905610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:52.906221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:52.908192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:52.919999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1338/2638] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:52.924355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:52.936276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac997-1c07-486d-8547-c2142300895d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:52.941052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:52.941747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:52.942066 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:52.942357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] PATH_INFO: `/v3/projects/06e6b2470d994405a6f145cd8cb681c7/users/e7f517fb13944c2cab5f381b3937e77a/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:53.095736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e6b2470d994405a6f145cd8cb681c7, user_id=e7f517fb13944c2cab5f381b3937e77a, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:53.098503 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:53.204129 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1301/2639] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:52 2022] PUT /identity/v3/projects/06e6b2470d994405a6f145cd8cb681c7/users/e7f517fb13944c2cab5f381b3937e77a/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:53.209207 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:53.219368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255bd0-60d6-404e-91b1-9e544b21cb10 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:53.278588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148865b0-af32-40b5-b83b-13739b91b99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:53.279169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148865b0-af32-40b5-b83b-13739b91b99e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:53.279484 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148865b0-af32-40b5-b83b-13739b91b99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:53.279773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148865b0-af32-40b5-b83b-13739b91b99e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:53.280471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148865b0-af32-40b5-b83b-13739b91b99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:53.283181 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148865b0-af32-40b5-b83b-13739b91b99e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:53.300450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1339/2640] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:53.305278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:53.347044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa314-a98b-4c5f-b3dc-72ebd63615b4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:53.360569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:53.361250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:53.361540 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:53.361817 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] PATH_INFO: `/v3/projects/06e6b2470d994405a6f145cd8cb681c7/users/e7f517fb13944c2cab5f381b3937e77a/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:53.364368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e6b2470d994405a6f145cd8cb681c7, user_id=e7f517fb13944c2cab5f381b3937e77a, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:53.367030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:53.379017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1302/2641] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:53 2022] PUT /identity/v3/projects/06e6b2470d994405a6f145cd8cb681c7/users/e7f517fb13944c2cab5f381b3937e77a/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:53.389749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:53.390158 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:53.390443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:53.431352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] MFA Rules not processed for user `e7f517fb13944c2cab5f381b3937e77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:53.432667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:53.523554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1340/2642] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:53.533569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:53.535700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:53.617967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723877ba-24a9-486a-ad9e-f9517a6d766f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:53.624789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:53.626228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:53.626512 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:53.626790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:53.640775 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:53.642234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:53.655365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1303/2643] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:54.074914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:54.076494 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:54.146267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1c99c-0d3a-435c-9a57-12cdd00d3eec None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:54.150802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:54.151492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:54.151776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:54.152061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] PATH_INFO: `/v3/users/89de93e6f9564e2cb73500e3d383e836` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:54.153239 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89de93e6f9564e2cb73500e3d383e836)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:54.155151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:54.174605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:54.365650 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89de93e6f9564e2cb73500e3d383e836'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:54.371704 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89de93e6f9564e2cb73500e3d383e836'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:54.372467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89de93e6f9564e2cb73500e3d383e836'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:54.373235 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89de93e6f9564e2cb73500e3d383e836'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:54.386756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89de93e6f9564e2cb73500e3d383e836'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:54.455405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1341/2644] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:00:54 2022] DELETE /identity/v3/users/89de93e6f9564e2cb73500e3d383e836 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:54.695558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:54.706370 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34da0dc-cf8b-4991-b46f-8c7857649761 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:54.744081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:54.745624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:54.755609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:54.766870 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:54.767771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:54.767989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:54.768253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] PATH_INFO: `/v3/users/27dcd3892ed84559ae338dff5afd0cc5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:54.769499 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=27dcd3892ed84559ae338dff5afd0cc5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:54.772450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:54.843981 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88255e3a-c0df-48a8-8969-766c53e16da7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:54.849241 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:54.849899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:54.850173 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:54.850454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] PATH_INFO: `/v3/projects/886baa7c44f641d287f911a81bebaa13` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:54.855313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=886baa7c44f641d287f911a81bebaa13)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:54.855313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.007573 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27dcd3892ed84559ae338dff5afd0cc5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.008044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886baa7c44f641d287f911a81bebaa13', 'request_id': 'req-ce8033c8-6f45-4fec-8ced-e9edfac06203'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.009925 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27dcd3892ed84559ae338dff5afd0cc5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.015865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27dcd3892ed84559ae338dff5afd0cc5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.118633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886baa7c44f641d287f911a81bebaa13', 'request_id': 'req-ce8033c8-6f45-4fec-8ced-e9edfac06203'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.124994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886baa7c44f641d287f911a81bebaa13', 'request_id': 'req-ce8033c8-6f45-4fec-8ced-e9edfac06203'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.139915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] The token cache is being invalidate because project 886baa7c44f641d287f911a81bebaa13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:00:55.140209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.140905 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ce8033c8-6f45-4fec-8ced-e9edfac06203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.142411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1342/2645] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:00:54 2022] DELETE /identity/v3/projects/886baa7c44f641d287f911a81bebaa13 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:55.147060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb099-ad1a-4cc1-af8e-2aca82cd532c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.147390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb099-ad1a-4cc1-af8e-2aca82cd532c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.147673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb099-ad1a-4cc1-af8e-2aca82cd532c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.167985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6b9cb099-ad1a-4cc1-af8e-2aca82cd532c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:55.169242 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6b9cb099-ad1a-4cc1-af8e-2aca82cd532c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:55.181329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27dcd3892ed84559ae338dff5afd0cc5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.188589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:55.195598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fbd495ac-1ad2-4c00-b9cf-038ab31677dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27dcd3892ed84559ae338dff5afd0cc5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:55.202030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1304/2646] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:00:54 2022] DELETE /identity/v3/users/27dcd3892ed84559ae338dff5afd0cc5 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:55.208648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.208940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.209249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.231860 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:55.233091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:55.240423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1343/2647] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:00:55.249358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb1afd-1f93-4655-b896-ca3860d2c18b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.249654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb1afd-1f93-4655-b896-ca3860d2c18b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.249939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb1afd-1f93-4655-b896-ca3860d2c18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.275269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e4bb1afd-1f93-4655-b896-ca3860d2c18b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:55.276414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e4bb1afd-1f93-4655-b896-ca3860d2c18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:55.298273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1305/2648] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:00:55.309064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.310589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:55.341403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1344/2649] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:00:55.345948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb1afd-1f93-4655-b896-ca3860d2c18b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.354965 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb1afd-1f93-4655-b896-ca3860d2c18b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.359755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.360390 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.360670 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.361141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.361642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:00:55.361948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.363728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.369047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52775996-2c82-4475-adbc-cb845bfba205 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.373106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.373730 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.374017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.374295 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.376229 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1345/2650] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:00:55.381070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.387052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.388548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.390506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bcb2e-5bf2-4770-8f1c-030f79df1136 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.395894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142791-4207-413a-9052-6362761950a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.396482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142791-4207-413a-9052-6362761950a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.396757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142791-4207-413a-9052-6362761950a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.397026 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142791-4207-413a-9052-6362761950a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.397686 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27142791-4207-413a-9052-6362761950a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.399118 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1306/2651] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:55.399605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27142791-4207-413a-9052-6362761950a7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.502443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1346/2652] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:55 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:55.507419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.517337 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814218de-fa64-4907-bf6f-4fc36b78226d service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.530239 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:55.584793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.585621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.585906 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.586218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.586875 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.588849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.728222 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1307/2653] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:55 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:55.732630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142791-4207-413a-9052-6362761950a7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.744153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142791-4207-413a-9052-6362761950a7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.749079 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.749770 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.750126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.750476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.751283 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.753110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.764659 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1347/2654] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:55.768793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.778972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b321c-0211-4b4e-aa61-412bb626f667 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.784670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.785230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.785769 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.786054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.786789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.788442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.799709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1308/2655] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:55.807304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.818177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71740b6-b7f1-44f3-82da-eb9660de8178 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.822815 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a94397-97b5-43a1-8ee1-32f43ae3800b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.823509 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94397-97b5-43a1-8ee1-32f43ae3800b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.823794 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94397-97b5-43a1-8ee1-32f43ae3800b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.824077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94397-97b5-43a1-8ee1-32f43ae3800b admin admin] PATH_INFO: `/v3/projects/b16344a7e86e413682e8ee99390ea2cd/users/f73b58a6c9e7420eb8c07c4e0ffb91fe/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.839568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a94397-97b5-43a1-8ee1-32f43ae3800b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16344a7e86e413682e8ee99390ea2cd, user_id=f73b58a6c9e7420eb8c07c4e0ffb91fe, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.842497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a94397-97b5-43a1-8ee1-32f43ae3800b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:55.922238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:55.924498 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:55.943790 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d1e98-bd02-4883-8ecc-58532f8d85b3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:55.949225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:55.949881 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:55.950261 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:55.950543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] PATH_INFO: `/v3/projects/93ad93a10d124e0b94cf0fb6f918d8ee` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:55.951815 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ad93a10d124e0b94cf0fb6f918d8ee)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:55.953968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:56.004842 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1348/2656] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:55 2022] PUT /identity/v3/projects/b16344a7e86e413682e8ee99390ea2cd/users/f73b58a6c9e7420eb8c07c4e0ffb91fe/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:56.015028 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a167f6-591a-4362-b16a-c67a52ab91a0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.015357 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a167f6-591a-4362-b16a-c67a52ab91a0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.015635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a167f6-591a-4362-b16a-c67a52ab91a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.043945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c2a167f6-591a-4362-b16a-c67a52ab91a0 None None] MFA Rules not processed for user `f73b58a6c9e7420eb8c07c4e0ffb91fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:56.045101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c2a167f6-591a-4362-b16a-c67a52ab91a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:56.089838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:56.100412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ad93a10d124e0b94cf0fb6f918d8ee', 'request_id': 'req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.107030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ad93a10d124e0b94cf0fb6f918d8ee', 'request_id': 'req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.111802 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1349/2657] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:00:56.117257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a167f6-591a-4362-b16a-c67a52ab91a0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:56.178612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a167f6-591a-4362-b16a-c67a52ab91a0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:56.181233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ad93a10d124e0b94cf0fb6f918d8ee', 'request_id': 'req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.192520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] The token cache is being invalidate because project 93ad93a10d124e0b94cf0fb6f918d8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:00:56.192812 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.193468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.194998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1309/2658] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:00:55 2022] DELETE /identity/v3/projects/93ad93a10d124e0b94cf0fb6f918d8ee => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:56.198919 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:56.200258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:56.243369 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:56.244003 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.244292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.244566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.245245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:56.247160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:56.266444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ec35c-ed7e-4b32-8da5-79ab1026980a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:56.271692 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:56.272331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.272613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.272886 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] PATH_INFO: `/v3/users/e7f517fb13944c2cab5f381b3937e77a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.274048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f517fb13944c2cab5f381b3937e77a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:56.276016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:56.298943 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:56.346679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1350/2659] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:56 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:56.351477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:56.353286 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:56.420684 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f566ce-adea-40bb-aa20-3329af8700a8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:56.427157 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:56.427805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.428134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.430360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.430360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:56.431145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:56.463524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f517fb13944c2cab5f381b3937e77a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.464190 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f517fb13944c2cab5f381b3937e77a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.471258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f517fb13944c2cab5f381b3937e77a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.562747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1351/2660] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:56 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:56.566836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:56.568512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:56.580792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74f856-6df2-4d04-ae9c-20f08e884c84 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:56.582863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f517fb13944c2cab5f381b3937e77a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.586219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:56.594535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.594535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.594535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.594535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:56.594535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:56.594535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:56.597933 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f517fb13944c2cab5f381b3937e77a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:56.601299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1352/2661] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:00:56.605360 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1310/2662] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:00:56 2022] DELETE /identity/v3/users/e7f517fb13944c2cab5f381b3937e77a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:56.605629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:56.621416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed3e90-6bd7-4a17-a20d-ae1d21097140 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:56.684327 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c3764-b5ba-403e-812e-e1cd46f280e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:56.684961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3764-b5ba-403e-812e-e1cd46f280e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.685247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3764-b5ba-403e-812e-e1cd46f280e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.685525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3764-b5ba-403e-812e-e1cd46f280e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.686221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c3764-b5ba-403e-812e-e1cd46f280e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:56.687851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c3764-b5ba-403e-812e-e1cd46f280e3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:56.698622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1353/2663] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:00:56.703278 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:56.714338 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70179f7a-75eb-4113-86d0-8b1da6d4f7a0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:56.719764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:56.719965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.720268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.720522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] PATH_INFO: `/v3/projects/a30aa696228e4f8e8e56089afed7d5a0/users/7b997f4de912410581e17c6cc686524a/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.735623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30aa696228e4f8e8e56089afed7d5a0, user_id=7b997f4de912410581e17c6cc686524a, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:56.738297 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:56.874141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1311/2664] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:56 2022] PUT /identity/v3/projects/a30aa696228e4f8e8e56089afed7d5a0/users/7b997f4de912410581e17c6cc686524a/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:56.885743 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:56.886205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:56.886599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:56.924104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] MFA Rules not processed for user `7b997f4de912410581e17c6cc686524a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:56.925467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.002746 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1354/2665] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:57.008124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.070853 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c23ec-a5ae-421c-9f98-81cdcb241b37 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.075467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.076083 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.076369 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.076647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.077317 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.079229 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.090148 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.100582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.107064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930e709-1bd8-4f95-b9e7-57a8293f28bb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.111658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.112322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.112634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.112951 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] PATH_INFO: `/v3/projects/06e6b2470d994405a6f145cd8cb681c7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.114710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e6b2470d994405a6f145cd8cb681c7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.116662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.217298 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1312/2666] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:57 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:57.221407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.263867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e6b2470d994405a6f145cd8cb681c7', 'request_id': 'req-ec7525c7-8623-459d-9a5d-c794dacbc0af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:57.281671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:57.285544 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47698-daf3-4204-8181-bb9e73fa7d94 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.342152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.342797 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.343122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.343475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.344179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.346221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.365129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e6b2470d994405a6f145cd8cb681c7', 'request_id': 'req-ec7525c7-8623-459d-9a5d-c794dacbc0af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:57.371321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e6b2470d994405a6f145cd8cb681c7', 'request_id': 'req-ec7525c7-8623-459d-9a5d-c794dacbc0af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:57.383786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] The token cache is being invalidate because project 06e6b2470d994405a6f145cd8cb681c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:00:57.384131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:57.384806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec7525c7-8623-459d-9a5d-c794dacbc0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:00:57.386347 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1355/2667] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:00:57 2022] DELETE /identity/v3/projects/06e6b2470d994405a6f145cd8cb681c7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:57.391930 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765219a-3ed4-4ec1-8255-cd4e9cedb7da None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.392262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765219a-3ed4-4ec1-8255-cd4e9cedb7da None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.392544 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765219a-3ed4-4ec1-8255-cd4e9cedb7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.401340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:57.413362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a765219a-3ed4-4ec1-8255-cd4e9cedb7da None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:57.419432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a765219a-3ed4-4ec1-8255-cd4e9cedb7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.423111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1356/2668] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:00:57.432166 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8a9c3-5e21-4413-866e-828bb9af336d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.432462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8a9c3-5e21-4413-866e-828bb9af336d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.432755 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8a9c3-5e21-4413-866e-828bb9af336d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.445595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1313/2669] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:57 2022] POST /identity/v3/users => generated 299 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:57.450717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.452737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-75c8a9c3-5e21-4413-866e-828bb9af336d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:57.452917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.453932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-75c8a9c3-5e21-4413-866e-828bb9af336d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.462179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1357/2670] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:57.464612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff60c-dca2-448f-a052-ab056ea41eb1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.466370 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8a9c3-5e21-4413-866e-828bb9af336d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.469825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.470635 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.471475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.471883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.472629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.474807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.477352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8a9c3-5e21-4413-866e-828bb9af336d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.482209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.482926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.483224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.483515 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.484236 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:00:57.484540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.485964 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1314/2671] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:00:57.489870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.490738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.500724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1358/2672] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:00:57.507331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b413c-95fd-414e-9c84-9d1e78c84111 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.507331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.507331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.507331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.507739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] PATH_INFO: `/v3/projects/e07af013d33041349c244ab376819593/users/2d486e3660054a64bd0a265b32465851/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.508183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.520133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533eaa4-fc3c-40a2-aa75-372ebdf16657 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.524735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07af013d33041349c244ab376819593, user_id=2d486e3660054a64bd0a265b32465851, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.525175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.525897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.526282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.526694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.527467 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.527597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.529722 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.616414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1315/2673] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:57 2022] PUT /identity/v3/projects/e07af013d33041349c244ab376819593/users/2d486e3660054a64bd0a265b32465851/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:57.620380 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.632510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1359/2674] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:57 2022] POST /identity/v3/projects => generated 401 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:57.636874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.682848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4255b5-f87d-4aae-a848-ba07a64868ff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.702703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581474c6-3dca-4e0e-98c0-1f4c3eb2bc7a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.707634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.708254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.708533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.708901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.709565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.711527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.741315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.741941 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.742230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.742510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.743213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.744843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.754578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1316/2675] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:00:57.758708 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.768410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ebcdb-50d6-4357-9c5f-797172a5c68e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.773903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b0f00-deb8-4485-bf29-c09a94c32154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.774552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b0f00-deb8-4485-bf29-c09a94c32154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.774846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b0f00-deb8-4485-bf29-c09a94c32154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.775209 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b0f00-deb8-4485-bf29-c09a94c32154 admin admin] PATH_INFO: `/v3/projects/e07af013d33041349c244ab376819593/users/2d486e3660054a64bd0a265b32465851/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.777382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b0f00-deb8-4485-bf29-c09a94c32154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07af013d33041349c244ab376819593, user_id=2d486e3660054a64bd0a265b32465851, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.780075 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b0f00-deb8-4485-bf29-c09a94c32154 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.791685 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1317/2676] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:57 2022] PUT /identity/v3/projects/e07af013d33041349c244ab376819593/users/2d486e3660054a64bd0a265b32465851/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:57.803169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.803510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.803774 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.818960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1360/2677] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:57 2022] POST /identity/v3/users => generated 301 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:57.820297 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.822371 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:57.830523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557820-e023-4926-806e-c2bc5e6b0a7d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.833931 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] MFA Rules not processed for user `2d486e3660054a64bd0a265b32465851`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:57.835400 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.886884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.888462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.888749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.889032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.889747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.891379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.901797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1361/2678] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:00:57.905969 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1318/2679] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:00:57.906617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.920026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.920700 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b2f8-4202-4965-9c0e-41cc00d89014 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.921667 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.926923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.927776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.928055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.928335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.929059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.930655 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.941511 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1362/2680] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:00:57.946176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.956482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ece8-04c9-41e5-93c2-e333c74dc875 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.962455 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.963146 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.963464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.963767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] PATH_INFO: `/v3/projects/98e3393a0a75427e964c318c700fb127/users/9766d202b87a44f98777c43364bf4d01/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.993137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e3393a0a75427e964c318c700fb127, user_id=9766d202b87a44f98777c43364bf4d01, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.993324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c85e72-1435-4c9b-84b3-aa9498186523 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.993496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.993859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbc7d7-ee62-4c17-92a9-42953a1405f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.993859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc7d7-ee62-4c17-92a9-42953a1405f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.993859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc7d7-ee62-4c17-92a9-42953a1405f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.993859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc7d7-ee62-4c17-92a9-42953a1405f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.004527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbc7d7-ee62-4c17-92a9-42953a1405f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:58.006120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbc7d7-ee62-4c17-92a9-42953a1405f1 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:58.016113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1319/2681] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:00:58.079667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1363/2682] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:57 2022] PUT /identity/v3/projects/98e3393a0a75427e964c318c700fb127/users/9766d202b87a44f98777c43364bf4d01/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:00:58.091928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee5e5b-df43-4354-a0f5-55372ea9eba6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.092450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee5e5b-df43-4354-a0f5-55372ea9eba6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.092731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee5e5b-df43-4354-a0f5-55372ea9eba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.123995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-10ee5e5b-df43-4354-a0f5-55372ea9eba6 None None] MFA Rules not processed for user `9766d202b87a44f98777c43364bf4d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:58.125496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-10ee5e5b-df43-4354-a0f5-55372ea9eba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:58.190727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1320/2683] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:58.195784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:58.205834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:58.261613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1940-a080-43cc-9b54-d7d38a68f1b4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:58.266560 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:58.267244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.267507 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.267790 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.270512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:58.271226 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:58.396884 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1364/2684] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:00:58 2022] POST /identity/v3/projects => generated 399 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:00:58.398275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee5e5b-df43-4354-a0f5-55372ea9eba6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:58.409684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee5e5b-df43-4354-a0f5-55372ea9eba6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:58.415026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:58.470648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:58.471323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.471600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.471882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.472560 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:58.474535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:58.581572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1321/2685] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:00:58 2022] POST /identity/v3/users => generated 306 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:00:58.585825 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:58.596257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188766b4-1b75-43f2-82c0-2fc8b66833b2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:58.601081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:58.601691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.601972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.602375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.603049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:58.604676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:58.617252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1365/2686] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:58.621549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:58.632482 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9622a-0733-4bb1-9689-bfe85e29a651 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:58.637165 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:58.637892 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.638202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.638508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] PATH_INFO: `/v3/projects/5f2c1b12b491402188d3d11521b0e69b/users/411f3c38860c4a1cbc2e2dc24ffdeb94/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.653202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2c1b12b491402188d3d11521b0e69b, user_id=411f3c38860c4a1cbc2e2dc24ffdeb94, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:58.656030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:58.752680 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1322/2687] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:58 2022] PUT /identity/v3/projects/5f2c1b12b491402188d3d11521b0e69b/users/411f3c38860c4a1cbc2e2dc24ffdeb94/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:58.764341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:58.767735 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2179c7-cd5b-4342-8f99-4c5e23044b8d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:58.825508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdace477-497d-45e5-bb38-06b63f0c25b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:58.826131 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdace477-497d-45e5-bb38-06b63f0c25b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.826477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdace477-497d-45e5-bb38-06b63f0c25b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.826692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdace477-497d-45e5-bb38-06b63f0c25b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.827416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdace477-497d-45e5-bb38-06b63f0c25b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:58.829000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdace477-497d-45e5-bb38-06b63f0c25b8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:58.838878 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1366/2688] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:00:58.843546 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:58.853764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d1757-718b-40f3-83cf-526828ed5bda admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:58.858264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:58.865828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.865828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.865828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] PATH_INFO: `/v3/projects/5f2c1b12b491402188d3d11521b0e69b/users/411f3c38860c4a1cbc2e2dc24ffdeb94/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.865828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2c1b12b491402188d3d11521b0e69b, user_id=411f3c38860c4a1cbc2e2dc24ffdeb94, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:58.865828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:58.874683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1323/2689] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:00:58 2022] PUT /identity/v3/projects/5f2c1b12b491402188d3d11521b0e69b/users/411f3c38860c4a1cbc2e2dc24ffdeb94/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:00:58.885909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:58.886205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:58.886498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:58.915300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] MFA Rules not processed for user `411f3c38860c4a1cbc2e2dc24ffdeb94`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:58.916550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:58.984176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1367/2690] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:00:58 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:00:58.995949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:59.065782 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0010d3b-abc2-4e2f-851e-71261d39f21c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:59.069749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:59.070385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:59.071397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:59.071397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:59.082742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:59.084398 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:59.101885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1324/2691] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:58 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:59.226135 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:59.229852 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:59.234493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:59.240697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc100b-1729-418a-8489-6f2616bfe4b7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:59.244361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:59.244980 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:59.245264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:59.245538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:59.247350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:59.308143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:59.309633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:59.318921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:59.319362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1368/2692] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:59.328810 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab7371-769c-452f-9c08-13ba129d9233 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:59.333461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:59.338866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:59.338866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:59.338866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:59.395615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:59.397155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:59.402326 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:59.406961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1325/2693] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:00:59 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:04.540421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:04.549538 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4a27b-00e7-4b8a-b609-fa07f81ac864 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:04.553673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a721d34-b321-4987-a423-9365f46ff188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:04.555332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a721d34-b321-4987-a423-9365f46ff188 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:04.555332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a721d34-b321-4987-a423-9365f46ff188 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:04.555332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a721d34-b321-4987-a423-9365f46ff188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:04.556656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6a721d34-b321-4987-a423-9365f46ff188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:04.617994 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a721d34-b321-4987-a423-9365f46ff188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:04.619473 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a721d34-b321-4987-a423-9365f46ff188 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:04.628419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1369/2694] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:04.806192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:04.872160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d57a67-5532-4194-b127-646e15bcab60 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:04.876874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:04.877662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:04.877951 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:04.878234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] PATH_INFO: `/v3/users/9766d202b87a44f98777c43364bf4d01` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:04.879456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9766d202b87a44f98777c43364bf4d01)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:04.881399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:05.130589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9766d202b87a44f98777c43364bf4d01'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:05.131273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9766d202b87a44f98777c43364bf4d01'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:05.138162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9766d202b87a44f98777c43364bf4d01'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:05.267551 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9766d202b87a44f98777c43364bf4d01'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:05.280282 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9766d202b87a44f98777c43364bf4d01'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:05.286506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1326/2695] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:04 2022] DELETE /identity/v3/users/9766d202b87a44f98777c43364bf4d01 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:05.291022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:05.291376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:05.291667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:05.311154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:05.312382 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:05.372053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1370/2696] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:05.382365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:05.383961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:05.444139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa634275-a7e2-47c5-8964-cd481a924e4f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:05.448649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:05.449763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:05.450269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:05.451054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:05.466656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:05.468160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:05.479160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1327/2697] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:05.941769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:05.941769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:05.951596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb2da6-2f1b-492d-8c93-918a316ac347 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:05.957043 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:05.957712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:05.958049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:05.958882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] PATH_INFO: `/v3/projects/98e3393a0a75427e964c318c700fb127` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:05.960151 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98e3393a0a75427e964c318c700fb127)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:05.962056 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:06.079649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:06.089044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98e3393a0a75427e964c318c700fb127', 'request_id': 'req-164d93d4-9f4c-40ab-aa2a-3a06687e145d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.176006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98e3393a0a75427e964c318c700fb127', 'request_id': 'req-164d93d4-9f4c-40ab-aa2a-3a06687e145d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.182866 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98e3393a0a75427e964c318c700fb127', 'request_id': 'req-164d93d4-9f4c-40ab-aa2a-3a06687e145d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.194778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] The token cache is being invalidate because project 98e3393a0a75427e964c318c700fb127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:06.195076 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.195848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.197383 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1371/2698] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:05 2022] DELETE /identity/v3/projects/98e3393a0a75427e964c318c700fb127 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:06.201416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:06.211547 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b515824-f0ef-4d3d-a482-4b5d42c60b26 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:06.230838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:06.272109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:06.272864 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:06.273062 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:06.273401 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] PATH_INFO: `/v3/users/411f3c38860c4a1cbc2e2dc24ffdeb94` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:06.274559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] RBAC: Authorizing `identity:delete_user(user_id=411f3c38860c4a1cbc2e2dc24ffdeb94)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:06.276536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:06.614139 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411f3c38860c4a1cbc2e2dc24ffdeb94'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.615610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411f3c38860c4a1cbc2e2dc24ffdeb94'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.622623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411f3c38860c4a1cbc2e2dc24ffdeb94'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.739123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411f3c38860c4a1cbc2e2dc24ffdeb94'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.752944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9f099562-db14-4c56-8c0a-c1226d58149a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411f3c38860c4a1cbc2e2dc24ffdeb94'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:06.758982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1328/2699] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:06 2022] DELETE /identity/v3/users/411f3c38860c4a1cbc2e2dc24ffdeb94 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:07.281555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:07.283322 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:07.297598 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d93d4-9f4c-40ab-aa2a-3a06687e145d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:07.358911 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:07.359584 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:07.359872 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:07.360150 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] PATH_INFO: `/v3/projects/5f2c1b12b491402188d3d11521b0e69b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:07.361356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2c1b12b491402188d3d11521b0e69b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:07.363295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:07.497049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2c1b12b491402188d3d11521b0e69b', 'request_id': 'req-2ba7c107-f986-492d-a57a-18be5a99b57f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:07.552897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2c1b12b491402188d3d11521b0e69b', 'request_id': 'req-2ba7c107-f986-492d-a57a-18be5a99b57f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:07.559839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2c1b12b491402188d3d11521b0e69b', 'request_id': 'req-2ba7c107-f986-492d-a57a-18be5a99b57f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:07.571745 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] The token cache is being invalidate because project 5f2c1b12b491402188d3d11521b0e69b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:07.572068 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:07.572741 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2ba7c107-f986-492d-a57a-18be5a99b57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:07.574317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1372/2700] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:07 2022] DELETE /identity/v3/projects/5f2c1b12b491402188d3d11521b0e69b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:07.580339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f971d-633d-4c20-a518-ef4ecee2fb3e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:07.580339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f971d-633d-4c20-a518-ef4ecee2fb3e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:07.580971 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f971d-633d-4c20-a518-ef4ecee2fb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:07.600155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-072f971d-633d-4c20-a518-ef4ecee2fb3e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:07.601353 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-072f971d-633d-4c20-a518-ef4ecee2fb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:07.672697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1329/2701] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:07.681182 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08a79-3916-4c9b-90a8-74248fe84c7a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:07.681468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08a79-3916-4c9b-90a8-74248fe84c7a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:07.681748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08a79-3916-4c9b-90a8-74248fe84c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:07.701992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e0f08a79-3916-4c9b-90a8-74248fe84c7a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:07.703225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e0f08a79-3916-4c9b-90a8-74248fe84c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:07.711253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1373/2702] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:01:07.715496 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f971d-633d-4c20-a518-ef4ecee2fb3e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:07.725111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f971d-633d-4c20-a518-ef4ecee2fb3e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:07.731215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:07.731962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:07.732253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:07.732528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:07.733226 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:01:07.733530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:07.735138 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:07.748563 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1330/2703] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:01:07.752910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08a79-3916-4c9b-90a8-74248fe84c7a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:07.762662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08a79-3916-4c9b-90a8-74248fe84c7a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:07.767590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:07.768069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:07.768350 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:07.768622 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:07.769288 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:07.771178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:07.879999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1374/2704] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:07 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:07.884221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:07.895116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c17e2a-f3a9-48a6-b493-b611776a28fa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:07.921896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:07.952694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:07.953396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:07.953688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:07.953967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:07.954791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:07.956753 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:08.057568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1331/2705] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:07 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:08.061558 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:08.072095 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6aef1-0946-4e45-a641-881193c924a6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:08.076654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:08.077333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.077586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.077850 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.078547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:08.080178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:08.090956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1375/2706] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:08.095019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:08.107323 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da3306-2efe-48bf-83b1-548164184f9d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:08.112609 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff777a-006e-405f-8beb-b46a15a80c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:08.115902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff777a-006e-405f-8beb-b46a15a80c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.115902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff777a-006e-405f-8beb-b46a15a80c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.115902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff777a-006e-405f-8beb-b46a15a80c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.115902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff777a-006e-405f-8beb-b46a15a80c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:08.116859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff777a-006e-405f-8beb-b46a15a80c4c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:08.127464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1332/2707] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:08.132161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:08.142379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d85ca-0c1a-4a3b-9740-74f8ddd25972 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:08.146998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:08.148425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.148726 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.149008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] PATH_INFO: `/v3/projects/ac0c655b37464de5a4933de79c454015/users/963697a9cdaa4cde994ee6b794c547b3/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.160752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:08.165184 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0c655b37464de5a4933de79c454015, user_id=963697a9cdaa4cde994ee6b794c547b3, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:08.168021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:08.280027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1376/2708] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:08 2022] PUT /identity/v3/projects/ac0c655b37464de5a4933de79c454015/users/963697a9cdaa4cde994ee6b794c547b3/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:08.297736 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.302316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.302483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.333437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] MFA Rules not processed for user `963697a9cdaa4cde994ee6b794c547b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:08.334776 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:08.412049 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1333/2709] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:08.424934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:08.426847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:08.486110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8772-283c-43d7-9cd4-2a49494952ca admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:08.489837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d945-4f23-43f1-bd88-023504d691e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:08.490485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d945-4f23-43f1-bd88-023504d691e9 service glance] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.490773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d945-4f23-43f1-bd88-023504d691e9 service glance] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.491054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d945-4f23-43f1-bd88-023504d691e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.499644 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d945-4f23-43f1-bd88-023504d691e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:08.501130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d945-4f23-43f1-bd88-023504d691e9 service glance] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:08.510060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1377/2710] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:09.055442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:09.056967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:09.128952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d07633-fff8-4ac3-90fe-587d9b78b3b7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:09.134536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:09.150202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:09.150202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:09.150202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] PATH_INFO: `/v3/users/f73b58a6c9e7420eb8c07c4e0ffb91fe` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:09.150202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f73b58a6c9e7420eb8c07c4e0ffb91fe)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:09.150202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:09.353710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:09.359511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f73b58a6c9e7420eb8c07c4e0ffb91fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:09.360157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f73b58a6c9e7420eb8c07c4e0ffb91fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:09.367610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f73b58a6c9e7420eb8c07c4e0ffb91fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:09.478719 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f73b58a6c9e7420eb8c07c4e0ffb91fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:09.492577 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f73b58a6c9e7420eb8c07c4e0ffb91fe'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:09.498651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1334/2711] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:09 2022] DELETE /identity/v3/users/f73b58a6c9e7420eb8c07c4e0ffb91fe => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:09.503405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:09.503926 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:09.504035 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:09.524082 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:09.525143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:09.587555 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1378/2712] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:09.597390 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:09.598806 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:09.657107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcf209-9c3c-4cd9-b8cc-a06f80aa79fe admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:09.661112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:09.661735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:09.662025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:09.662315 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:09.672274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:09.673781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:09.684486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1335/2713] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:10.262265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:10.264177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:10.280399 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208d661-72db-471d-a68a-c3f49e3394ca None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:10.290228 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:10.291262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:10.291553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:10.291830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] PATH_INFO: `/v3/projects/b16344a7e86e413682e8ee99390ea2cd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:10.293137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16344a7e86e413682e8ee99390ea2cd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:10.298127 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:10.418570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16344a7e86e413682e8ee99390ea2cd', 'request_id': 'req-50c8f9fc-0bb4-4136-8de0-f92336158556'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.534044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16344a7e86e413682e8ee99390ea2cd', 'request_id': 'req-50c8f9fc-0bb4-4136-8de0-f92336158556'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.540105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16344a7e86e413682e8ee99390ea2cd', 'request_id': 'req-50c8f9fc-0bb4-4136-8de0-f92336158556'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.551414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] The token cache is being invalidate because project b16344a7e86e413682e8ee99390ea2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:10.551710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.552377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.554395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1379/2714] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:10 2022] DELETE /identity/v3/projects/b16344a7e86e413682e8ee99390ea2cd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:10.558446 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:10.568406 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26973e1-f866-4e01-850f-3120172afe76 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:10.629930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:10.630858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:10.631152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:10.631462 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] PATH_INFO: `/v3/users/7b997f4de912410581e17c6cc686524a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:10.633440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b997f4de912410581e17c6cc686524a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:10.635569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:10.806863 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b997f4de912410581e17c6cc686524a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.807541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b997f4de912410581e17c6cc686524a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.817649 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b997f4de912410581e17c6cc686524a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.823248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:10.884221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b997f4de912410581e17c6cc686524a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.897054 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b997f4de912410581e17c6cc686524a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:10.902962 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1336/2715] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:10 2022] DELETE /identity/v3/users/7b997f4de912410581e17c6cc686524a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:11.377353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:11.378782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:11.392601 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8f9fc-0bb4-4136-8de0-f92336158556 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:11.405219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:11.450646 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:11.451346 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:11.451624 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:11.452129 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] PATH_INFO: `/v3/projects/a30aa696228e4f8e8e56089afed7d5a0` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:11.453608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30aa696228e4f8e8e56089afed7d5a0)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:11.456568 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:11.598925 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30aa696228e4f8e8e56089afed7d5a0', 'request_id': 'req-8843d18b-d5ce-4855-8566-604130515597'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:11.687489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30aa696228e4f8e8e56089afed7d5a0', 'request_id': 'req-8843d18b-d5ce-4855-8566-604130515597'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:11.693467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30aa696228e4f8e8e56089afed7d5a0', 'request_id': 'req-8843d18b-d5ce-4855-8566-604130515597'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:11.704885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] The token cache is being invalidate because project a30aa696228e4f8e8e56089afed7d5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:11.705187 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:11.705936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:11.707532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1380/2716] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:11 2022] DELETE /identity/v3/projects/a30aa696228e4f8e8e56089afed7d5a0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:11.711643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:11.713079 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:11.726341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55576c74-a25f-4470-8e04-690f84e67fe6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:11.790311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:11.790989 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:11.791305 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:11.791586 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] PATH_INFO: `/v3/users/2d486e3660054a64bd0a265b32465851` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:11.792746 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d486e3660054a64bd0a265b32465851)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:11.794718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:12.015644 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d486e3660054a64bd0a265b32465851'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.016308 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d486e3660054a64bd0a265b32465851'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.024090 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d486e3660054a64bd0a265b32465851'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.101929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d486e3660054a64bd0a265b32465851'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.116043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-61dd71e0-f88a-4ccf-9870-7abc316736a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d486e3660054a64bd0a265b32465851'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.122029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1337/2717] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:11 2022] DELETE /identity/v3/users/2d486e3660054a64bd0a265b32465851 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:01:12.583154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:12.584985 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:12.599817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d18b-d5ce-4855-8566-604130515597 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:12.660252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:12.660910 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:12.661192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:12.661469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] PATH_INFO: `/v3/projects/e07af013d33041349c244ab376819593` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:12.662630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07af013d33041349c244ab376819593)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:12.665099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:12.776574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:12.777882 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07af013d33041349c244ab376819593', 'request_id': 'req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.878234 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07af013d33041349c244ab376819593', 'request_id': 'req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.884764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07af013d33041349c244ab376819593', 'request_id': 'req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.896353 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] The token cache is being invalidate because project e07af013d33041349c244ab376819593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:12.896645 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.897298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4f7656c2-2709-49b0-b566-5d4a3f9d1ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:12.898796 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1381/2718] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:12 2022] DELETE /identity/v3/projects/e07af013d33041349c244ab376819593 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:12.904910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba01c4a-fdf4-43d3-857d-26f60375eaf3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:12.905430 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba01c4a-fdf4-43d3-857d-26f60375eaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:12.905558 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba01c4a-fdf4-43d3-857d-26f60375eaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:12.926653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3ba01c4a-fdf4-43d3-857d-26f60375eaf3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:12.927910 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3ba01c4a-fdf4-43d3-857d-26f60375eaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:12.983684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:12.993626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1338/2719] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:13.002062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd419-5467-457c-bfcb-df644585636d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.002421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd419-5467-457c-bfcb-df644585636d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.002762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd419-5467-457c-bfcb-df644585636d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.022521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3dffd419-5467-457c-bfcb-df644585636d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:13.023778 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3dffd419-5467-457c-bfcb-df644585636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:13.031826 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1382/2720] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:13.036281 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba01c4a-fdf4-43d3-857d-26f60375eaf3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.045939 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba01c4a-fdf4-43d3-857d-26f60375eaf3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.050705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.053949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.053949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.053949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.053949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:01:13.053949 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.055142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:13.067000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1339/2721] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:01:13.071967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffd419-5467-457c-bfcb-df644585636d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.081579 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffd419-5467-457c-bfcb-df644585636d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.086361 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.086969 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.087300 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.087585 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.088264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.090168 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:13.196517 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1383/2722] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:13 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:13.199658 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.211293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262152bb-71a5-452a-9ef4-88eda207d652 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.271904 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.272741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.273025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.273301 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.273979 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.275930 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:13.368991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1340/2723] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:13 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:01:13.373219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.383623 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ac75d-905e-412a-ae4f-d8540a60771d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.388230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.388860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.389169 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.389442 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.390128 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.391742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:13.401773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1384/2724] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:13.405765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.417266 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80885cc3-838d-4beb-aace-408ee0965ff9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.422993 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d469f94-1ed6-4537-b845-ff9cfd0cc280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.423841 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d469f94-1ed6-4537-b845-ff9cfd0cc280 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.424132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d469f94-1ed6-4537-b845-ff9cfd0cc280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.424581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d469f94-1ed6-4537-b845-ff9cfd0cc280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.425314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d469f94-1ed6-4537-b845-ff9cfd0cc280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.427212 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d469f94-1ed6-4537-b845-ff9cfd0cc280 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:13.440686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1341/2725] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:13.444840 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.456411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896ba88-caa4-400c-ba51-9d99833b738b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.461763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.462447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.462732 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.463011 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] PATH_INFO: `/v3/projects/3678ea3765634c5d88af6ee19a710ed0/users/774649213dc84cddbaf46656697865cf/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.478329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3678ea3765634c5d88af6ee19a710ed0, user_id=774649213dc84cddbaf46656697865cf, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.480999 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:13.562635 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1385/2726] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:13 2022] PUT /identity/v3/projects/3678ea3765634c5d88af6ee19a710ed0/users/774649213dc84cddbaf46656697865cf/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:13.574987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df869-523f-49ef-ba31-b8b486d30796 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.575290 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df869-523f-49ef-ba31-b8b486d30796 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.575576 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df869-523f-49ef-ba31-b8b486d30796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.604128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e43df869-523f-49ef-ba31-b8b486d30796 None None] MFA Rules not processed for user `774649213dc84cddbaf46656697865cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:13.605357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e43df869-523f-49ef-ba31-b8b486d30796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:13.676920 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1342/2727] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:13.682085 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.745078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135254-72b0-4cc1-8747-ac5fa6d19219 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.749656 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.750271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.750553 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.750827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.751531 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.753505 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:13.877828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1386/2728] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:13 2022] POST /identity/v3/projects => generated 367 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:01:13.881883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df869-523f-49ef-ba31-b8b486d30796 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:13.892559 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df869-523f-49ef-ba31-b8b486d30796 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:13.942085 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:13.952351 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:13.960720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:13.960720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:13.960720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:13.960720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:13.960720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:14.036836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1343/2729] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:13 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:14.041421 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:14.051819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e209745-4615-4f02-9268-515f90fa0205 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:14.057126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:14.057771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:14.058088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:14.058367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:14.059073 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:14.060703 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:14.071315 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1387/2730] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:14.075487 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:14.085484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beb6b4-d96f-412b-b761-70803e8cd7de admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:14.090141 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:14.090808 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:14.091095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:14.091404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] PATH_INFO: `/v3/projects/2ae85c2d3eb44e628431ded67f5df14c/users/c87b2e74a15e45108e1118dc50892177/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:14.106779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae85c2d3eb44e628431ded67f5df14c, user_id=c87b2e74a15e45108e1118dc50892177, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:14.109506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:14.185758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1344/2731] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:14 2022] PUT /identity/v3/projects/2ae85c2d3eb44e628431ded67f5df14c/users/c87b2e74a15e45108e1118dc50892177/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:14.189705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:14.200194 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705da29-744a-4ae4-9623-3b3130d720db admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:14.251548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:14.264209 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13a69e-c446-4004-ae81-1bf7c447e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:14.264835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13a69e-c446-4004-ae81-1bf7c447e070 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:14.265120 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13a69e-c446-4004-ae81-1bf7c447e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:14.265396 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13a69e-c446-4004-ae81-1bf7c447e070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:14.266093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13a69e-c446-4004-ae81-1bf7c447e070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:14.267730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13a69e-c446-4004-ae81-1bf7c447e070 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:14.278051 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1388/2732] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:14.282230 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:14.296048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c3d28-3e32-463f-91b7-620feeed9cc1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:14.298825 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:14.299294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:14.299581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:14.300026 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] PATH_INFO: `/v3/projects/2ae85c2d3eb44e628431ded67f5df14c/users/c87b2e74a15e45108e1118dc50892177/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:14.302015 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae85c2d3eb44e628431ded67f5df14c, user_id=c87b2e74a15e45108e1118dc50892177, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:14.305437 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:14.316960 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1345/2733] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:14 2022] PUT /identity/v3/projects/2ae85c2d3eb44e628431ded67f5df14c/users/c87b2e74a15e45108e1118dc50892177/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:14.332988 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:14.333279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:14.333564 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:14.362975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] MFA Rules not processed for user `c87b2e74a15e45108e1118dc50892177`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:14.364253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:14.432927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1389/2734] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:14.444985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:14.446659 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:14.505061 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb59b1-d8bb-4fbb-b7e8-74fa12b08be6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:14.509606 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:14.510268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:14.510717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:14.511006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:14.520978 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:14.522477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:14.531913 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1346/2735] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:14.782422 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:14.784561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:14.799728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286bfc0-e1d0-4a32-9036-2103cb267704 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:14.803501 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57f6c2-ed19-4109-85e5-2b90b3becb4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:14.804669 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57f6c2-ed19-4109-85e5-2b90b3becb4a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:14.804956 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57f6c2-ed19-4109-85e5-2b90b3becb4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:14.805233 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57f6c2-ed19-4109-85e5-2b90b3becb4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:14.861721 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57f6c2-ed19-4109-85e5-2b90b3becb4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:14.863159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57f6c2-ed19-4109-85e5-2b90b3becb4a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:14.872769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1390/2736] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:21.704752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:21.770752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f992-686e-463a-848d-d86481ae296a service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:21.776021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:21.776561 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:21.776845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:21.777122 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] PATH_INFO: `/v3/users/774649213dc84cddbaf46656697865cf` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:21.778286 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=774649213dc84cddbaf46656697865cf)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:21.780334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:21.958150 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:21.959838 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774649213dc84cddbaf46656697865cf'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:21.960618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774649213dc84cddbaf46656697865cf'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:21.967699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774649213dc84cddbaf46656697865cf'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.048106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774649213dc84cddbaf46656697865cf'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.062419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774649213dc84cddbaf46656697865cf'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.068612 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1347/2737] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:21 2022] DELETE /identity/v3/users/774649213dc84cddbaf46656697865cf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:22.073845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:22.074321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:22.074428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:22.107345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:22.108592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:22.161380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:22.171435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1391/2738] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:22.181623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:22.183036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:22.249441 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e213-cb40-4f54-a8e0-ba0aa2023d5d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:22.253255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:22.253883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:22.254156 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:22.254428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:22.264095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:22.265527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:22.274768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1348/2739] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:22.672499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:22.673927 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:22.687329 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a788095-5a17-455b-8a90-fd23ee4599e0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:22.692175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:22.692831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:22.693115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:22.693396 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] PATH_INFO: `/v3/projects/3678ea3765634c5d88af6ee19a710ed0` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:22.694582 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3678ea3765634c5d88af6ee19a710ed0)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:22.696524 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:22.785203 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3678ea3765634c5d88af6ee19a710ed0', 'request_id': 'req-3387b494-9ec2-4b1b-beff-076498d3b9b5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.867880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3678ea3765634c5d88af6ee19a710ed0', 'request_id': 'req-3387b494-9ec2-4b1b-beff-076498d3b9b5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.874355 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3678ea3765634c5d88af6ee19a710ed0', 'request_id': 'req-3387b494-9ec2-4b1b-beff-076498d3b9b5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.885476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] The token cache is being invalidate because project 3678ea3765634c5d88af6ee19a710ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:22.885788 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.886408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:22.887957 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1392/2740] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:22 2022] DELETE /identity/v3/projects/3678ea3765634c5d88af6ee19a710ed0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:22.891923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:22.901485 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a049c5-1439-42f8-b114-0d80e2159c40 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:22.958849 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:22.959527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:22.959830 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:22.960110 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] PATH_INFO: `/v3/users/c87b2e74a15e45108e1118dc50892177` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:22.961268 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87b2e74a15e45108e1118dc50892177)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:22.963219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:23.182828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87b2e74a15e45108e1118dc50892177'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:23.183568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87b2e74a15e45108e1118dc50892177'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:23.191257 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87b2e74a15e45108e1118dc50892177'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:23.195045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:23.276946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87b2e74a15e45108e1118dc50892177'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:23.291101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f55da5eb-b9e7-4618-8877-d8006487b864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87b2e74a15e45108e1118dc50892177'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:23.297214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1349/2741] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:22 2022] DELETE /identity/v3/users/c87b2e74a15e45108e1118dc50892177 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:23.734936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:23.736453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:23.750009 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387b494-9ec2-4b1b-beff-076498d3b9b5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:23.808402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:23.809050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:23.809336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:23.809617 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] PATH_INFO: `/v3/projects/2ae85c2d3eb44e628431ded67f5df14c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:23.810772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae85c2d3eb44e628431ded67f5df14c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:23.812714 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:23.923923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae85c2d3eb44e628431ded67f5df14c', 'request_id': 'req-4c7460cb-d246-4756-b73a-853ca5111de3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:23.996547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae85c2d3eb44e628431ded67f5df14c', 'request_id': 'req-4c7460cb-d246-4756-b73a-853ca5111de3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:24.002950 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae85c2d3eb44e628431ded67f5df14c', 'request_id': 'req-4c7460cb-d246-4756-b73a-853ca5111de3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:24.014823 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] The token cache is being invalidate because project 2ae85c2d3eb44e628431ded67f5df14c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:24.015117 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:24.015793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4c7460cb-d246-4756-b73a-853ca5111de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:24.017443 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1393/2742] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:23 2022] DELETE /identity/v3/projects/2ae85c2d3eb44e628431ded67f5df14c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:24.022706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df01d4-4da3-48fb-ac8a-f9b05dc5d43b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.023296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df01d4-4da3-48fb-ac8a-f9b05dc5d43b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.023296 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df01d4-4da3-48fb-ac8a-f9b05dc5d43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.043514 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-91df01d4-4da3-48fb-ac8a-f9b05dc5d43b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:24.044718 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-91df01d4-4da3-48fb-ac8a-f9b05dc5d43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:24.104155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1350/2743] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:24.112211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed66f-84c0-4c60-883f-1cad0f2fab44 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.112510 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed66f-84c0-4c60-883f-1cad0f2fab44 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.112792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed66f-84c0-4c60-883f-1cad0f2fab44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.132253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-749ed66f-84c0-4c60-883f-1cad0f2fab44 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:24.133461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-749ed66f-84c0-4c60-883f-1cad0f2fab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:24.141343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1394/2744] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:24.145648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df01d4-4da3-48fb-ac8a-f9b05dc5d43b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.155999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df01d4-4da3-48fb-ac8a-f9b05dc5d43b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:24.160745 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:24.161347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.161627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.161898 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.162602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:01:24.163584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:24.165180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:24.176976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1351/2745] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:01:24.181126 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed66f-84c0-4c60-883f-1cad0f2fab44 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.186752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:24.190636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed66f-84c0-4c60-883f-1cad0f2fab44 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:24.195528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:24.196142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.196453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.196725 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.197386 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:24.199299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:24.311748 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1395/2746] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:24 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:24.316024 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.326713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f115a-ffb3-4654-a841-319abb7e5d92 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:24.385859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:24.386619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.386952 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.387358 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.387895 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:24.389836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:24.479858 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1352/2747] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:24 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:24.484090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.495211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daac85d-f79d-4ac4-8772-cf7450fab408 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:24.499975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:24.500595 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.500879 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.501159 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.501853 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:24.503490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:24.515063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1396/2748] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:24.519130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.529518 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4f784-0812-429b-bafe-2684bcc81ba1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:24.535359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62708d3b-4e84-46e2-984c-a2957ec725fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:24.535991 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62708d3b-4e84-46e2-984c-a2957ec725fe admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.536279 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62708d3b-4e84-46e2-984c-a2957ec725fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.536565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62708d3b-4e84-46e2-984c-a2957ec725fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.537269 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62708d3b-4e84-46e2-984c-a2957ec725fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:24.539548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62708d3b-4e84-46e2-984c-a2957ec725fe admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:24.550713 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1353/2749] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:24.555798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.566164 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a24f6f-a21e-43cf-a0ce-4317f2944bb1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:24.570881 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:24.571592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.571877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.572162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] PATH_INFO: `/v3/projects/1e6a5195792b424b830257ebb4b931a2/users/2f1225a24c1343c6b36e2934290cacf6/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.587175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6a5195792b424b830257ebb4b931a2, user_id=2f1225a24c1343c6b36e2934290cacf6, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:24.589863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:24.671833 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1397/2750] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:24 2022] PUT /identity/v3/projects/1e6a5195792b424b830257ebb4b931a2/users/2f1225a24c1343c6b36e2934290cacf6/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:24.684384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494e6f1-4105-4deb-bce1-e961b25c03ae None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.684679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494e6f1-4105-4deb-bce1-e961b25c03ae None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.684957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494e6f1-4105-4deb-bce1-e961b25c03ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.716656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2494e6f1-4105-4deb-bce1-e961b25c03ae None None] MFA Rules not processed for user `2f1225a24c1343c6b36e2934290cacf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:24.717859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2494e6f1-4105-4deb-bce1-e961b25c03ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:24.750143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:24.781793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1354/2751] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:24.787358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.848685 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc545f-0750-4605-98ae-8bc93055475c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:24.853454 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:24.854064 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:24.854342 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:24.854616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:24.855293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:24.857183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:24.947432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1398/2752] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:24 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:24.951632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494e6f1-4105-4deb-bce1-e961b25c03ae None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:24.962316 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494e6f1-4105-4deb-bce1-e961b25c03ae None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:25.024889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:25.025530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:25.025786 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:25.026064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:25.026731 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:25.028683 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:25.128123 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1355/2753] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:24 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:25.132206 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:25.142804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c8457-036a-496e-81e8-f4166d414d0d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:25.147608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:25.148225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:25.148513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:25.148791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:25.149587 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:25.151211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:25.163814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1399/2754] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:25.168113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:25.178424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076abb86-cb61-452e-8255-1d6d2fef2e67 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:25.183569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:25.184233 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:25.184522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:25.184802 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] PATH_INFO: `/v3/projects/da8442ec7af24a579e8603197a11d83c/users/babfc008dd064024a356c48d089ff516/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:25.199568 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8442ec7af24a579e8603197a11d83c, user_id=babfc008dd064024a356c48d089ff516, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:25.202224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:25.271457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1356/2755] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:25 2022] PUT /identity/v3/projects/da8442ec7af24a579e8603197a11d83c/users/babfc008dd064024a356c48d089ff516/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:01:25.275724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:25.286282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd51b4-1c7a-4eae-8352-f3d262e6f83a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:25.343460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee838a3-1156-4f9f-be8d-9236f1f0cd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:25.344090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee838a3-1156-4f9f-be8d-9236f1f0cd7e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:25.344375 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee838a3-1156-4f9f-be8d-9236f1f0cd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:25.344666 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee838a3-1156-4f9f-be8d-9236f1f0cd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:25.345356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee838a3-1156-4f9f-be8d-9236f1f0cd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:25.346966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee838a3-1156-4f9f-be8d-9236f1f0cd7e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:25.357569 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1400/2756] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:25.361720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:25.372897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b063fc-996c-45b7-b23e-441b92c38930 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:25.377567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:25.378277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:25.378522 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:25.378800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] PATH_INFO: `/v3/projects/da8442ec7af24a579e8603197a11d83c/users/babfc008dd064024a356c48d089ff516/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:25.381177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8442ec7af24a579e8603197a11d83c, user_id=babfc008dd064024a356c48d089ff516, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:25.383915 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:25.394331 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1357/2757] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:25 2022] PUT /identity/v3/projects/da8442ec7af24a579e8603197a11d83c/users/babfc008dd064024a356c48d089ff516/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:25.404975 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eecd-36cc-4f8e-aa80-8f92121523e4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:25.405273 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eecd-36cc-4f8e-aa80-8f92121523e4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:25.405547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eecd-36cc-4f8e-aa80-8f92121523e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:25.434411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2424eecd-36cc-4f8e-aa80-8f92121523e4 None None] MFA Rules not processed for user `babfc008dd064024a356c48d089ff516`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:25.435638 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2424eecd-36cc-4f8e-aa80-8f92121523e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:25.440653 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:25.510565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1401/2758] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:01:25.518733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:25.520465 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:25.592011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6abee-f0d6-499f-b65d-74b698d74d12 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:25.595968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480617b-41b2-4986-8a9d-d2db4263f64f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:25.596593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480617b-41b2-4986-8a9d-d2db4263f64f service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:25.596877 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480617b-41b2-4986-8a9d-d2db4263f64f service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:25.597151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480617b-41b2-4986-8a9d-d2db4263f64f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:25.607619 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480617b-41b2-4986-8a9d-d2db4263f64f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:25.609106 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480617b-41b2-4986-8a9d-d2db4263f64f service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:25.619732 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1358/2759] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:26.723092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424eecd-36cc-4f8e-aa80-8f92121523e4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:26.793138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424eecd-36cc-4f8e-aa80-8f92121523e4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:26.797589 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:26.798244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:26.798527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:26.798800 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] PATH_INFO: `/v3/users/2f1225a24c1343c6b36e2934290cacf6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:26.800052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1225a24c1343c6b36e2934290cacf6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:26.802008 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:27.081784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1225a24c1343c6b36e2934290cacf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:27.089841 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1225a24c1343c6b36e2934290cacf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:27.090539 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1225a24c1343c6b36e2934290cacf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:27.091264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1225a24c1343c6b36e2934290cacf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:27.103922 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1225a24c1343c6b36e2934290cacf6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:27.231018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1402/2760] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:26 2022] DELETE /identity/v3/users/2f1225a24c1343c6b36e2934290cacf6 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:27.235618 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:27.236095 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:27.236179 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:27.254637 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:27.255873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:27.268397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:27.316097 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1359/2761] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:27.327417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:27.328844 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:27.356832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:27.388088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493b8a5-443d-4cdb-a249-23727f9a3d5c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:27.392405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:27.393027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:27.393312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:27.393586 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:27.403175 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:27.405752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:27.415219 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1403/2762] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:27.897700 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:27.899524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:27.914231 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1dadbb-345d-40b9-a5a1-d41c806a4731 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:27.919234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:27.920012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:27.920224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:27.920502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] PATH_INFO: `/v3/projects/1e6a5195792b424b830257ebb4b931a2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:27.921728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6a5195792b424b830257ebb4b931a2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:27.923670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:28.044368 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6a5195792b424b830257ebb4b931a2', 'request_id': 'req-dc9be81c-81a6-40fe-b29c-da8750fffbce'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.051119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6a5195792b424b830257ebb4b931a2', 'request_id': 'req-dc9be81c-81a6-40fe-b29c-da8750fffbce'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.119130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6a5195792b424b830257ebb4b931a2', 'request_id': 'req-dc9be81c-81a6-40fe-b29c-da8750fffbce'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.132852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] The token cache is being invalidate because project 1e6a5195792b424b830257ebb4b931a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:28.133410 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.134022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.136650 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1360/2763] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:27 2022] DELETE /identity/v3/projects/1e6a5195792b424b830257ebb4b931a2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:28.139895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:28.151492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0dad-306d-4c59-8a4b-bb587a6e4f2b service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:28.209886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:28.210545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:28.210831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:28.211116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] PATH_INFO: `/v3/users/babfc008dd064024a356c48d089ff516` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:28.212324 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] RBAC: Authorizing `identity:delete_user(user_id=babfc008dd064024a356c48d089ff516)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:28.214244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:28.466851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'babfc008dd064024a356c48d089ff516'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.473691 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'babfc008dd064024a356c48d089ff516'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.474380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'babfc008dd064024a356c48d089ff516'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.474995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'babfc008dd064024a356c48d089ff516'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.487845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89882241-bfb2-4e64-80ee-714e46c2f20c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'babfc008dd064024a356c48d089ff516'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:28.593525 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1404/2764] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:28 2022] DELETE /identity/v3/users/babfc008dd064024a356c48d089ff516 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:29.020866 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:29.022423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:29.037161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9be81c-81a6-40fe-b29c-da8750fffbce admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:29.105938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:29.106601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.106884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.107161 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] PATH_INFO: `/v3/projects/da8442ec7af24a579e8603197a11d83c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.108372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8442ec7af24a579e8603197a11d83c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:29.110288 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:29.218130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:29.221162 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8442ec7af24a579e8603197a11d83c', 'request_id': 'req-30bc0881-1c81-480c-8f03-1968b3a1fdb5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:29.228373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8442ec7af24a579e8603197a11d83c', 'request_id': 'req-30bc0881-1c81-480c-8f03-1968b3a1fdb5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:29.310581 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8442ec7af24a579e8603197a11d83c', 'request_id': 'req-30bc0881-1c81-480c-8f03-1968b3a1fdb5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:29.322490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] The token cache is being invalidate because project da8442ec7af24a579e8603197a11d83c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:29.322800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:29.323594 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-30bc0881-1c81-480c-8f03-1968b3a1fdb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:29.325258 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1361/2765] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:29 2022] DELETE /identity/v3/projects/da8442ec7af24a579e8603197a11d83c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:29.330262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6d1d3-186e-4100-a8bf-5ac6ab2651d3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.330752 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6d1d3-186e-4100-a8bf-5ac6ab2651d3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.330855 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6d1d3-186e-4100-a8bf-5ac6ab2651d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.350966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-aef6d1d3-186e-4100-a8bf-5ac6ab2651d3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:29.352202 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aef6d1d3-186e-4100-a8bf-5ac6ab2651d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:29.397740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:29.416287 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1405/2766] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:01:29.425451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da1c6-0424-461d-adbb-47ec57a4ea89 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.425818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da1c6-0424-461d-adbb-47ec57a4ea89 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.426119 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da1c6-0424-461d-adbb-47ec57a4ea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.447598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-244da1c6-0424-461d-adbb-47ec57a4ea89 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:29.448800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-244da1c6-0424-461d-adbb-47ec57a4ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:29.456737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1362/2767] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:29.461147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6d1d3-186e-4100-a8bf-5ac6ab2651d3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:29.470960 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6d1d3-186e-4100-a8bf-5ac6ab2651d3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:29.475682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:29.476295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.479773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.479773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.479773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:01:29.479773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:29.479773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:29.492184 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1406/2768] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:01:29.496747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244da1c6-0424-461d-adbb-47ec57a4ea89 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:29.506375 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244da1c6-0424-461d-adbb-47ec57a4ea89 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:29.511809 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:29.512425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.512798 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.513140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.513805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:29.515805 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:29.598418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1363/2769] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:29 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:29.603113 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:29.613081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b35a2-c643-4bea-bc74-09a6612e2a99 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:29.670019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:29.670641 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.670920 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.671223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.671887 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:29.673786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:29.750737 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1407/2770] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:29 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:29.755208 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:29.765492 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc453a-9e56-4c7d-a90c-11d0ecf6073c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:29.770016 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:29.770621 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.770903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.771202 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.771907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:29.773580 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:29.783513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1364/2771] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:29.792097 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:29.802192 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247f380-bb3f-45b4-b962-69a995a5fa7d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:29.807469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbacf0-ad21-450c-bb0d-c8f0baac456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:29.808067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbacf0-ad21-450c-bb0d-c8f0baac456a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.808347 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbacf0-ad21-450c-bb0d-c8f0baac456a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.808651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbacf0-ad21-450c-bb0d-c8f0baac456a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.809337 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbacf0-ad21-450c-bb0d-c8f0baac456a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:29.810921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbacf0-ad21-450c-bb0d-c8f0baac456a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:29.821715 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1408/2772] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:29.826043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:29.835985 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf0676-963b-4a5e-beba-1689be2b6d91 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:29.840622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:29.841277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.841564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.841845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] PATH_INFO: `/v3/projects/99804f6475924bfc94a9be1a35df8388/users/935bdef547c441aebb25b56665f2397a/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.857067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99804f6475924bfc94a9be1a35df8388, user_id=935bdef547c441aebb25b56665f2397a, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:29.859709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:29.945356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1365/2773] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:29 2022] PUT /identity/v3/projects/99804f6475924bfc94a9be1a35df8388/users/935bdef547c441aebb25b56665f2397a/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:29.956220 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58723a96-526c-4d13-ab4e-990880ff78c3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:29.956508 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58723a96-526c-4d13-ab4e-990880ff78c3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:29.956782 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58723a96-526c-4d13-ab4e-990880ff78c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:29.985485 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-58723a96-526c-4d13-ab4e-990880ff78c3 None None] MFA Rules not processed for user `935bdef547c441aebb25b56665f2397a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:29.986766 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-58723a96-526c-4d13-ab4e-990880ff78c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:30.051550 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1409/2774] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:30.056253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:30.117506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2db71-ff8b-4603-bc9e-74146e3620d0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:30.122148 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:30.122760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.123042 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.123414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.124091 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:30.125970 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:30.228253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1366/2775] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:30 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:01:30.233111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58723a96-526c-4d13-ab4e-990880ff78c3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:30.247138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58723a96-526c-4d13-ab4e-990880ff78c3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:30.290450 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:30.310856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:30.311523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.311810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.312088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.312773 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:30.314754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:30.432316 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1410/2776] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:30 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:01:30.436473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:30.447903 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a7268-8918-46f4-9831-bd7870f162f6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:30.452564 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:30.453186 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.453474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.453819 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.454613 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:30.456679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:30.467710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1367/2777] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:30.471744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:30.482768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b24b5f-91a3-4382-a472-272cb857efec admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:30.487472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a2163-3537-478f-964f-34d6c1c378ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:30.488063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a2163-3537-478f-964f-34d6c1c378ec admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.488343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a2163-3537-478f-964f-34d6c1c378ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.488625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a2163-3537-478f-964f-34d6c1c378ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.489313 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a2163-3537-478f-964f-34d6c1c378ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:30.490916 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a2163-3537-478f-964f-34d6c1c378ec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:30.501892 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1411/2778] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:30.506250 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:30.516521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96908bfb-b70d-48b9-8ce6-659dff0f77f4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:30.521345 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:30.522007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.522293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.522572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] PATH_INFO: `/v3/projects/93fda348b74e40f8bc7e054fffd645d7/users/0fb6447e89f94c0ab6408d8ea80e9739/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.538693 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93fda348b74e40f8bc7e054fffd645d7, user_id=0fb6447e89f94c0ab6408d8ea80e9739, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:30.541419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:30.623113 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1368/2779] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:30 2022] PUT /identity/v3/projects/93fda348b74e40f8bc7e054fffd645d7/users/0fb6447e89f94c0ab6408d8ea80e9739/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:30.634190 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708bfe-7c30-4357-b613-7962bc525cf8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.634490 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708bfe-7c30-4357-b613-7962bc525cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.634777 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708bfe-7c30-4357-b613-7962bc525cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.663453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b9708bfe-7c30-4357-b613-7962bc525cf8 None None] MFA Rules not processed for user `0fb6447e89f94c0ab6408d8ea80e9739`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:30.664711 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9708bfe-7c30-4357-b613-7962bc525cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:30.731765 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1412/2780] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:30.736207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:30.758526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:30.801497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b74a6-724a-4cfd-aef2-a9ae9160049f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:30.806795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:30.807436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.807725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.808004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.808681 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:30.810596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:30.897481 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1369/2781] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:30 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:01:30.901600 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708bfe-7c30-4357-b613-7962bc525cf8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:30.912832 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708bfe-7c30-4357-b613-7962bc525cf8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:30.969000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:30.969619 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:30.969899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:30.970174 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:30.970845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:30.972786 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:31.060147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1413/2782] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:30 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:01:31.064273 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:31.075715 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31424148-1bf1-4c34-9440-baa4b3e39500 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:31.080515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:31.081131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:31.081414 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:31.081695 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:31.082408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:31.084131 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:31.095047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1370/2783] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:31.099535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:31.110328 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d18bbb-1f74-4297-a7b7-2655072bc06b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:31.115024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:31.115698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:31.116093 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:31.116368 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] PATH_INFO: `/v3/projects/bb2fec7f6ef94fd1a1c5c8a51e4a7e94/users/20dd1d8dce264009a71b7e19c2ff0a32/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:31.132954 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2fec7f6ef94fd1a1c5c8a51e4a7e94, user_id=20dd1d8dce264009a71b7e19c2ff0a32, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:31.135662 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:31.250814 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1414/2784] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:31 2022] PUT /identity/v3/projects/bb2fec7f6ef94fd1a1c5c8a51e4a7e94/users/20dd1d8dce264009a71b7e19c2ff0a32/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:31.255011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:31.265461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4634f-8d41-4db1-ae79-53b4ead75292 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:31.324490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f3179-c138-4189-9b12-4e029b89707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:31.325171 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f3179-c138-4189-9b12-4e029b89707e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:31.325456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f3179-c138-4189-9b12-4e029b89707e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:31.325878 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f3179-c138-4189-9b12-4e029b89707e admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:31.326648 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f3179-c138-4189-9b12-4e029b89707e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:31.328359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f3179-c138-4189-9b12-4e029b89707e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:31.338328 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1371/2785] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:31.342602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:31.353451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa618612-3dee-4b12-a01a-51f3459fcd7e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:31.358296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:31.358945 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:31.359253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:31.359532 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] PATH_INFO: `/v3/projects/bb2fec7f6ef94fd1a1c5c8a51e4a7e94/users/20dd1d8dce264009a71b7e19c2ff0a32/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:31.361489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2fec7f6ef94fd1a1c5c8a51e4a7e94, user_id=20dd1d8dce264009a71b7e19c2ff0a32, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:31.364144 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:31.370934 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:31.375557 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1415/2786] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:31 2022] PUT /identity/v3/projects/bb2fec7f6ef94fd1a1c5c8a51e4a7e94/users/20dd1d8dce264009a71b7e19c2ff0a32/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:31.394511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e6a2e-1404-431b-b706-68591fafaeae None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:31.394780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e6a2e-1404-431b-b706-68591fafaeae None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:31.395103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e6a2e-1404-431b-b706-68591fafaeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:31.425764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dc5e6a2e-1404-431b-b706-68591fafaeae None None] MFA Rules not processed for user `20dd1d8dce264009a71b7e19c2ff0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:31.427044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dc5e6a2e-1404-431b-b706-68591fafaeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:31.500072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1372/2787] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:31.515083 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:31.517070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:31.584052 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5d881-4cf2-4db0-8d21-2d237a2949bb admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:31.588365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:31.589013 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:31.589320 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:31.589599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:31.650626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:31.652090 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:31.661593 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1416/2788] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:33.728623 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e6a2e-1404-431b-b706-68591fafaeae None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:33.738550 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e6a2e-1404-431b-b706-68591fafaeae None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:33.742219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:33.742837 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:33.743128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:33.743460 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:33.755698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:33.805126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:33.806735 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:33.817687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1373/2789] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:35.257574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:35.267125 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb1440-9a28-451f-8aa4-bf9b7cfe68e3 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:35.270962 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19669ad7-558c-4484-addb-f07447834643 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:35.271906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19669ad7-558c-4484-addb-f07447834643 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:35.271906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19669ad7-558c-4484-addb-f07447834643 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:35.272784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19669ad7-558c-4484-addb-f07447834643 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:35.282364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19669ad7-558c-4484-addb-f07447834643 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:35.284118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19669ad7-558c-4484-addb-f07447834643 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:35.294321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1417/2790] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:43.333821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.398846 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420da92-378f-4fe1-a885-94106dbc9c93 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.404396 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.405053 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.405341 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.405626 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] PATH_INFO: `/v3/users/935bdef547c441aebb25b56665f2397a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.406847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=935bdef547c441aebb25b56665f2397a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.408865 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.654397 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935bdef547c441aebb25b56665f2397a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:43.655048 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935bdef547c441aebb25b56665f2397a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:43.662453 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935bdef547c441aebb25b56665f2397a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:43.730953 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935bdef547c441aebb25b56665f2397a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:43.745484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935bdef547c441aebb25b56665f2397a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:43.752673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1374/2791] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:43 2022] DELETE /identity/v3/users/935bdef547c441aebb25b56665f2397a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:43.756991 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.757479 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.757567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.777224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:43.778356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:43.838466 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1418/2792] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:43.849077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.850652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:43.895176 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:43.924768 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c1c54-0a51-4948-ba39-ebf3f9d68c44 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.928422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.929041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.929329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.929607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.940484 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.942030 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.951823 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1375/2793] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:44.456017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:44.460941 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:44.469430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49c537-b4fd-412c-abbb-a3ac27089ec7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:44.473984 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:44.474621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:44.474904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:44.475177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] PATH_INFO: `/v3/projects/99804f6475924bfc94a9be1a35df8388` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:44.476358 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99804f6475924bfc94a9be1a35df8388)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:44.478343 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:44.581245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:44.590679 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99804f6475924bfc94a9be1a35df8388', 'request_id': 'req-de424ff6-7994-40bf-9a42-b82702b97c51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:44.679637 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99804f6475924bfc94a9be1a35df8388', 'request_id': 'req-de424ff6-7994-40bf-9a42-b82702b97c51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:44.686415 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99804f6475924bfc94a9be1a35df8388', 'request_id': 'req-de424ff6-7994-40bf-9a42-b82702b97c51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:44.698416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] The token cache is being invalidate because project 99804f6475924bfc94a9be1a35df8388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:44.698791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:44.699438 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:44.700997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1419/2794] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:44 2022] DELETE /identity/v3/projects/99804f6475924bfc94a9be1a35df8388 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:44.706662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:44.716982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3402e-888f-4f81-b282-e5b3755f9b2a service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:44.779993 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:44.780673 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:44.780959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:44.781243 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] PATH_INFO: `/v3/users/0fb6447e89f94c0ab6408d8ea80e9739` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:44.782436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb6447e89f94c0ab6408d8ea80e9739)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:44.784703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:45.011425 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb6447e89f94c0ab6408d8ea80e9739'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.012132 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb6447e89f94c0ab6408d8ea80e9739'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.020427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb6447e89f94c0ab6408d8ea80e9739'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.103418 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb6447e89f94c0ab6408d8ea80e9739'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.120114 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb6447e89f94c0ab6408d8ea80e9739'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.126307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1376/2795] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:44 2022] DELETE /identity/v3/users/0fb6447e89f94c0ab6408d8ea80e9739 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:45.621774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:45.621774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:45.621774 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de424ff6-7994-40bf-9a42-b82702b97c51 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:45.711041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:45.711769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:45.712055 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:45.712336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] PATH_INFO: `/v3/projects/93fda348b74e40f8bc7e054fffd645d7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:45.713562 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=93fda348b74e40f8bc7e054fffd645d7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:45.715527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:45.838992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93fda348b74e40f8bc7e054fffd645d7', 'request_id': 'req-4dc04e97-4471-445e-b514-6d7f50fabafd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.936468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93fda348b74e40f8bc7e054fffd645d7', 'request_id': 'req-4dc04e97-4471-445e-b514-6d7f50fabafd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.942989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93fda348b74e40f8bc7e054fffd645d7', 'request_id': 'req-4dc04e97-4471-445e-b514-6d7f50fabafd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.955069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] The token cache is being invalidate because project 93fda348b74e40f8bc7e054fffd645d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:45.955387 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.956024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:45.957526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1420/2796] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:45 2022] DELETE /identity/v3/projects/93fda348b74e40f8bc7e054fffd645d7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:45.961749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:45.963256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:45.976662 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb8891-94c1-42d7-8379-f80d9cf847cb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:46.032669 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:46.040442 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:46.041145 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:46.041439 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:46.041725 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] PATH_INFO: `/v3/users/20dd1d8dce264009a71b7e19c2ff0a32` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:46.043064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20dd1d8dce264009a71b7e19c2ff0a32)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:46.045055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:46.268333 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20dd1d8dce264009a71b7e19c2ff0a32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:46.269137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20dd1d8dce264009a71b7e19c2ff0a32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:46.276651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20dd1d8dce264009a71b7e19c2ff0a32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:46.370513 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20dd1d8dce264009a71b7e19c2ff0a32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:46.383791 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20dd1d8dce264009a71b7e19c2ff0a32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:46.390158 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1377/2797] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:45 2022] DELETE /identity/v3/users/20dd1d8dce264009a71b7e19c2ff0a32 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:46.648025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:46.650742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:46.664410 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc04e97-4471-445e-b514-6d7f50fabafd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:46.723000 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:46.742561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:46.742561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:46.742561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:46.742561 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] PATH_INFO: `/v3/users/963697a9cdaa4cde994ee6b794c547b3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:46.742903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] RBAC: Authorizing `identity:delete_user(user_id=963697a9cdaa4cde994ee6b794c547b3)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:46.742903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:47.037376 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963697a9cdaa4cde994ee6b794c547b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.044728 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963697a9cdaa4cde994ee6b794c547b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.045439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963697a9cdaa4cde994ee6b794c547b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.046044 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963697a9cdaa4cde994ee6b794c547b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.058631 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-edffa5e4-daa8-4457-a0bf-e32efbd3deba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963697a9cdaa4cde994ee6b794c547b3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.153882 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.155533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:47.169037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7c7a-8eaa-4453-b58e-da401f611156 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.173871 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:47.174654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.175249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.175249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] PATH_INFO: `/v3/projects/bb2fec7f6ef94fd1a1c5c8a51e4a7e94` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.176312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2fec7f6ef94fd1a1c5c8a51e4a7e94)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:47.178347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:47.181425 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1421/2798] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:46 2022] DELETE /identity/v3/users/963697a9cdaa4cde994ee6b794c547b3 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:47.185949 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af9259-658a-423d-a36b-eb673585756a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.186289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af9259-658a-423d-a36b-eb673585756a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.186736 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af9259-658a-423d-a36b-eb673585756a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.207153 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d9af9259-658a-423d-a36b-eb673585756a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:47.208403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d9af9259-658a-423d-a36b-eb673585756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:47.275077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1422/2799] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:47.285004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af9259-658a-423d-a36b-eb673585756a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.286447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d9af9259-658a-423d-a36b-eb673585756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:47.304505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2fec7f6ef94fd1a1c5c8a51e4a7e94', 'request_id': 'req-d48160e6-c77b-45c9-801b-77e8867116a8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.311428 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2fec7f6ef94fd1a1c5c8a51e4a7e94', 'request_id': 'req-d48160e6-c77b-45c9-801b-77e8867116a8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.345240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af9259-658a-423d-a36b-eb673585756a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.378498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:47.399077 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2db2fe-886f-4316-9a0a-3d36aa563480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:47.399632 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2db2fe-886f-4316-9a0a-3d36aa563480 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.399921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2db2fe-886f-4316-9a0a-3d36aa563480 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.400217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2db2fe-886f-4316-9a0a-3d36aa563480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.405050 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2fec7f6ef94fd1a1c5c8a51e4a7e94', 'request_id': 'req-d48160e6-c77b-45c9-801b-77e8867116a8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.416956 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] The token cache is being invalidate because project bb2fec7f6ef94fd1a1c5c8a51e4a7e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:47.417255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.417921 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d48160e6-c77b-45c9-801b-77e8867116a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:47.419584 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1378/2800] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:47 2022] DELETE /identity/v3/projects/bb2fec7f6ef94fd1a1c5c8a51e4a7e94 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:01:47.424757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208626a-9ad9-4252-b20b-a2b2db39f95d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.425043 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208626a-9ad9-4252-b20b-a2b2db39f95d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.425324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208626a-9ad9-4252-b20b-a2b2db39f95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.447733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6208626a-9ad9-4252-b20b-a2b2db39f95d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:47.449140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6208626a-9ad9-4252-b20b-a2b2db39f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:47.473403 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2db2fe-886f-4316-9a0a-3d36aa563480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:47.475027 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2db2fe-886f-4316-9a0a-3d36aa563480 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:47.486540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1423/2801] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:47.507721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:47.520827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1379/2802] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:01:47.530134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1beba-b0a6-4d49-b982-6b09b59e69cb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.530419 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1beba-b0a6-4d49-b982-6b09b59e69cb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.530836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1beba-b0a6-4d49-b982-6b09b59e69cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.553267 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e3f1beba-b0a6-4d49-b982-6b09b59e69cb None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:47.554668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e3f1beba-b0a6-4d49-b982-6b09b59e69cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:47.564225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1424/2803] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:47.569041 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208626a-9ad9-4252-b20b-a2b2db39f95d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.579265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208626a-9ad9-4252-b20b-a2b2db39f95d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.584362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:47.587046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.587046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.587046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.587046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:01:47.587046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:47.588344 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:47.603234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1380/2804] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:01:47.606002 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1beba-b0a6-4d49-b982-6b09b59e69cb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.616160 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1beba-b0a6-4d49-b982-6b09b59e69cb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.621677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:47.622338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.622626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.622908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.623761 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:47.625680 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:47.719727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1425/2805] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:01:47 2022] POST /identity/v3/projects => generated 399 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:47.724194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.737543 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c335-88c3-4b67-9d22-ebab670782da admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.818046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:47.818717 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.819006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.819318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.820006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:47.822051 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:47.926942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1381/2806] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:01:47 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:47.931709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.943038 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13d240-d2db-4673-9cb8-f9d968ca768a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.949071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:47.949863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.950215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.950591 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.951408 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:47.953566 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.955611 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:47.957217 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:47.967837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1426/2807] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:47.971663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:47.984336 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8bb78-3f9f-4b27-8954-5020a4df0a95 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.989271 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:47.990116 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:47.990427 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:47.990536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e2a74-4929-49b0-bda5-b579e18cf7cb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:47.990797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:47.991513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:47.993215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:48.000703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:48.000703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:48.000703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:48.000703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] PATH_INFO: `/v3/projects/ac0c655b37464de5a4933de79c454015` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:48.001347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0c655b37464de5a4933de79c454015)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:48.001347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:48.008105 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1427/2808] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:48.009261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:48.019364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8a9d0-080e-4c27-b4b3-f1af96f01f84 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:48.024674 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bc9f9-9e4b-4490-9729-09a23d28ba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:48.034936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bc9f9-9e4b-4490-9729-09a23d28ba04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:48.034936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bc9f9-9e4b-4490-9729-09a23d28ba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:48.034936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bc9f9-9e4b-4490-9729-09a23d28ba04 admin admin] PATH_INFO: `/v3/projects/c9d86eb94cb448d79eb1029eb16ee24c/users/c83071d8db4e4b34832fccda1ff8c738/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:48.042744 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bc9f9-9e4b-4490-9729-09a23d28ba04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d86eb94cb448d79eb1029eb16ee24c, user_id=c83071d8db4e4b34832fccda1ff8c738, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:48.045430 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bc9f9-9e4b-4490-9729-09a23d28ba04 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:48.129830 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1428/2809] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:01:48 2022] PUT /identity/v3/projects/c9d86eb94cb448d79eb1029eb16ee24c/users/c83071d8db4e4b34832fccda1ff8c738/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:01:48.142496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592092a-9b7f-4f92-a5ef-7523945f5179 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:48.143030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592092a-9b7f-4f92-a5ef-7523945f5179 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:48.143414 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592092a-9b7f-4f92-a5ef-7523945f5179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:48.144932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0c655b37464de5a4933de79c454015', 'request_id': 'req-8edee66b-e0a7-4574-8302-d154ecf7ced3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:48.153255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0c655b37464de5a4933de79c454015', 'request_id': 'req-8edee66b-e0a7-4574-8302-d154ecf7ced3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:48.176222 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b592092a-9b7f-4f92-a5ef-7523945f5179 None None] MFA Rules not processed for user `c83071d8db4e4b34832fccda1ff8c738`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:48.177469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b592092a-9b7f-4f92-a5ef-7523945f5179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:48.222507 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0c655b37464de5a4933de79c454015', 'request_id': 'req-8edee66b-e0a7-4574-8302-d154ecf7ced3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:48.238950 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] The token cache is being invalidate because project ac0c655b37464de5a4933de79c454015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:48.239395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:48.240068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:48.241835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:48.242197 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1382/2810] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:47 2022] DELETE /identity/v3/projects/ac0c655b37464de5a4933de79c454015 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:48.253556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1429/2811] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:48.269022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:48.270703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:48.348031 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edee66b-e0a7-4574-8302-d154ecf7ced3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:48.352624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7515e-80db-4ff1-8be4-82e224ea8be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:48.353468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7515e-80db-4ff1-8be4-82e224ea8be6 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:48.353675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7515e-80db-4ff1-8be4-82e224ea8be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:48.354025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7515e-80db-4ff1-8be4-82e224ea8be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:48.356157 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e3f7515e-80db-4ff1-8be4-82e224ea8be6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:48.368836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7515e-80db-4ff1-8be4-82e224ea8be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:48.370490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7515e-80db-4ff1-8be4-82e224ea8be6 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:48.376726 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:48.381443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1383/2812] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:57.612989 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592092a-9b7f-4f92-a5ef-7523945f5179 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:57.675661 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592092a-9b7f-4f92-a5ef-7523945f5179 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:57.680246 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:57.680893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:57.681179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:57.681463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] PATH_INFO: `/v3/users/c83071d8db4e4b34832fccda1ff8c738` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:57.683137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83071d8db4e4b34832fccda1ff8c738)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:57.685094 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:57.913447 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83071d8db4e4b34832fccda1ff8c738'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.921572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83071d8db4e4b34832fccda1ff8c738'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.922296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83071d8db4e4b34832fccda1ff8c738'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.923032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83071d8db4e4b34832fccda1ff8c738'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.935807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83071d8db4e4b34832fccda1ff8c738'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.032665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1430/2813] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:01:57 2022] DELETE /identity/v3/users/c83071d8db4e4b34832fccda1ff8c738 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:58.037552 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:58.038092 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:58.038180 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:58.056889 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:58.058154 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:58.115905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1384/2814] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:58.126019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:58.127535 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:58.197931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b52838-3153-4359-9c51-af22fb0e592f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:58.197931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:58.197931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:58.197931 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:58.198272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:58.206963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:58.208431 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:58.218103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1431/2815] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:58.613947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:58.615464 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:58.628185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12e1d-5e5a-48ca-8a3f-c1b95eb7fbe9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:58.633064 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:58.633733 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:58.634039 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:58.634411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] PATH_INFO: `/v3/projects/c9d86eb94cb448d79eb1029eb16ee24c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:58.635538 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d86eb94cb448d79eb1029eb16ee24c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:58.637506 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:58.743705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d86eb94cb448d79eb1029eb16ee24c', 'request_id': 'req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.750169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d86eb94cb448d79eb1029eb16ee24c', 'request_id': 'req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.820001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d86eb94cb448d79eb1029eb16ee24c', 'request_id': 'req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.831697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] The token cache is being invalidate because project c9d86eb94cb448d79eb1029eb16ee24c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:58.832000 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.832723 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.834274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1385/2816] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:01:58 2022] DELETE /identity/v3/projects/c9d86eb94cb448d79eb1029eb16ee24c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:08.213756 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:08.829515 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:33.570312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:33.571771 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:33.593007 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afc87d-eac0-4c95-9966-87cbf0328ee7 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:33.643335 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:33.643963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:33.644252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:33.644533 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:33.646304 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:33.708597 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:33.710020 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:33.719723 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1432/2817] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:37.384687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:37.386126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:37.459472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef0cb-a5f4-46d0-aa15-0b8a7c4034ae admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:37.464771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:37.465399 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:37.465688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:37.466004 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:37.467868 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:37.480336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:37.481975 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:37.489467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:37.490917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:37.494828 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1386/2818] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:37.515528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86c9e-a982-482e-b523-ad3de9656c73 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:37.521367 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:37.521594 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:37.521821 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:37.522081 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] PATH_INFO: `/v3/users/d2ee2b6dd6f647179d2af999b3e62788` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:37.523318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ee2b6dd6f647179d2af999b3e62788)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:37.525248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:37.710521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ee2b6dd6f647179d2af999b3e62788'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:37.717610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ee2b6dd6f647179d2af999b3e62788'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:37.718291 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ee2b6dd6f647179d2af999b3e62788'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:37.718979 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ee2b6dd6f647179d2af999b3e62788'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:37.731255 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c3b91b1-4beb-4324-9b45-3a7f8f3b2b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ee2b6dd6f647179d2af999b3e62788'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:37.795147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1433/2819] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:02:37 2022] DELETE /identity/v3/users/d2ee2b6dd6f647179d2af999b3e62788 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:38.057128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.068035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef4006-eea3-4eaf-b304-1d37030e6bd9 service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.128686 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.129343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.129696 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.129885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] PATH_INFO: `/v3/projects/0dc1c45a26e84a3bbc06d52fec42e332` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.131052 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc1c45a26e84a3bbc06d52fec42e332)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.133047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.273189 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc1c45a26e84a3bbc06d52fec42e332', 'request_id': 'req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.285461 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc1c45a26e84a3bbc06d52fec42e332', 'request_id': 'req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.386047 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc1c45a26e84a3bbc06d52fec42e332', 'request_id': 'req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.398572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] The token cache is being invalidate because project 0dc1c45a26e84a3bbc06d52fec42e332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:38.398767 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.399364 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ef5be9aa-e257-40c7-9554-34cfbcc5b9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.401116 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1387/2820] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:02:38 2022] DELETE /identity/v3/projects/0dc1c45a26e84a3bbc06d52fec42e332 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:38.407577 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a90b-1264-4de3-8607-7b26c0b35a2a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.407861 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a90b-1264-4de3-8607-7b26c0b35a2a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.408141 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a90b-1264-4de3-8607-7b26c0b35a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.428384 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-25c1a90b-1264-4de3-8607-7b26c0b35a2a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:38.429612 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-25c1a90b-1264-4de3-8607-7b26c0b35a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:38.438890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:38.491865 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1434/2821] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:38.499938 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e36325-e48a-425d-883a-383ee085ef3b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.500234 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e36325-e48a-425d-883a-383ee085ef3b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.500519 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e36325-e48a-425d-883a-383ee085ef3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.520524 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f1e36325-e48a-425d-883a-383ee085ef3b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:38.522240 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f1e36325-e48a-425d-883a-383ee085ef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:38.531598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1388/2822] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:38.537974 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a90b-1264-4de3-8607-7b26c0b35a2a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.547971 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a90b-1264-4de3-8607-7b26c0b35a2a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.552530 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.553134 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.553416 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.553688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.554405 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:38.554733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.556395 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.568154 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1435/2823] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:38.576844 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e36325-e48a-425d-883a-383ee085ef3b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.582444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e36325-e48a-425d-883a-383ee085ef3b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.587789 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.588422 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.588694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.588972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.589640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.591600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.604103 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:38.719419 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1389/2824] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:02:38 2022] POST /identity/v3/projects => generated 391 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:38.722115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.732489 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801828d1-732a-4ff0-8a7c-36989da901f6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.789633 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.790262 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.790599 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.791111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.792649 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.794652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.892230 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1436/2825] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:02:38 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:38.897077 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.907508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d3266-6140-453a-8a56-645f6af6d33e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.912843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.913457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.913744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.914022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.914722 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.916359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.926516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1390/2826] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:38.930523 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.940657 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec288f83-a215-45c0-83cb-8e9129e46e88 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.945883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ccf61-a226-4081-b244-faf362be2a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.946504 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ccf61-a226-4081-b244-faf362be2a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.946775 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ccf61-a226-4081-b244-faf362be2a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.947053 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ccf61-a226-4081-b244-faf362be2a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.947797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ccf61-a226-4081-b244-faf362be2a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.949423 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ccf61-a226-4081-b244-faf362be2a0a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.960828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1437/2827] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:38.965690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.975843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4441e4-e5b5-48e9-a29b-98f4b105022d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.980652 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.981312 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.981601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.981888 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] PATH_INFO: `/v3/projects/b0dd5932001f4f618130198706a14d8b/users/63067046807f42c5b701b0dbb6a15171/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.996965 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0dd5932001f4f618130198706a14d8b, user_id=63067046807f42c5b701b0dbb6a15171, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.999651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.066210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1391/2828] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:02:38 2022] PUT /identity/v3/projects/b0dd5932001f4f618130198706a14d8b/users/63067046807f42c5b701b0dbb6a15171/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:39.076689 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d081b-7ca3-4b08-9e41-e30126705523 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.076986 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d081b-7ca3-4b08-9e41-e30126705523 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.077270 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d081b-7ca3-4b08-9e41-e30126705523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.105966 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-929d081b-7ca3-4b08-9e41-e30126705523 None None] MFA Rules not processed for user `63067046807f42c5b701b0dbb6a15171`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:39.107217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-929d081b-7ca3-4b08-9e41-e30126705523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:39.171556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1438/2829] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:39.175947 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.242256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27ca56-aafb-4bdd-9a4a-fb26a82d31bb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.246977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.247622 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.247907 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.248182 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.248857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.250747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.336884 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1392/2830] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:02:39 2022] POST /identity/v3/projects => generated 393 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:39.341341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d081b-7ca3-4b08-9e41-e30126705523 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.354047 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d081b-7ca3-4b08-9e41-e30126705523 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.358902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:39.414772 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.415464 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.415749 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.416057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.416715 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.418658 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.515513 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1439/2831] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:02:39 2022] POST /identity/v3/users => generated 303 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:39.519677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.531248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0e874-9741-4d41-9052-b6b2be8b3d07 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.536147 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.537121 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.537407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.537711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.538408 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.540177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.551008 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1393/2832] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:39.555453 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.566360 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f3947-c012-4052-833d-02242af915d0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.571554 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.572213 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.572493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.572767 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] PATH_INFO: `/v3/projects/29c1ff9e8a9e4443a33e9ba60d1d947f/users/cda50cefd1ec430cac2b23bf9310ef13/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.588183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c1ff9e8a9e4443a33e9ba60d1d947f, user_id=cda50cefd1ec430cac2b23bf9310ef13, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.590817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.666867 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1440/2833] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:02:39 2022] PUT /identity/v3/projects/29c1ff9e8a9e4443a33e9ba60d1d947f/users/cda50cefd1ec430cac2b23bf9310ef13/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:39.670857 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.681130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f240-d5bd-4429-81cb-a996fe1687e7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.736630 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa58879-2b78-45e4-a5a4-9fe996b3a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.737248 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa58879-2b78-45e4-a5a4-9fe996b3a966 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.737539 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa58879-2b78-45e4-a5a4-9fe996b3a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.737816 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa58879-2b78-45e4-a5a4-9fe996b3a966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.738505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa58879-2b78-45e4-a5a4-9fe996b3a966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.740734 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa58879-2b78-45e4-a5a4-9fe996b3a966 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.750210 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1394/2834] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:39.754396 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.765477 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec55620-479f-4b9e-8515-816b730ddade admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.770088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.770760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.771050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.771364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] PATH_INFO: `/v3/projects/29c1ff9e8a9e4443a33e9ba60d1d947f/users/cda50cefd1ec430cac2b23bf9310ef13/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.773309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c1ff9e8a9e4443a33e9ba60d1d947f, user_id=cda50cefd1ec430cac2b23bf9310ef13, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.776001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.786521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1441/2835] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:02:39 2022] PUT /identity/v3/projects/29c1ff9e8a9e4443a33e9ba60d1d947f/users/cda50cefd1ec430cac2b23bf9310ef13/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:39.854219 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.854510 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.854793 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.881986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] MFA Rules not processed for user `cda50cefd1ec430cac2b23bf9310ef13`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:39.883214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:39.895387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:39.951310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1395/2836] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:39.962697 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.964448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:40.025614 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d087ca2-5a87-430e-aabd-0c4cb2d91e6a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.029163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.029783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.030063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.030334 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.040332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.041856 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.051799 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1442/2837] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:40.099832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:40.100287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:40.112671 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b2bca-3896-4d76-af60-b993d2ea290b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.116424 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19518c4a-ce0a-4443-a817-d489c7b22367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.117044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518c4a-ce0a-4443-a817-d489c7b22367 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.117327 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518c4a-ce0a-4443-a817-d489c7b22367 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.117605 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518c4a-ce0a-4443-a817-d489c7b22367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.174190 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19518c4a-ce0a-4443-a817-d489c7b22367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.175679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19518c4a-ce0a-4443-a817-d489c7b22367 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.184641 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1396/2838] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:44.787006 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:44.851163 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa5fe2-d2a4-4e63-b4a2-950ca63d4771 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:44.855897 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:44.856546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:44.856834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:44.857114 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] PATH_INFO: `/v3/users/63067046807f42c5b701b0dbb6a15171` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:44.858292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63067046807f42c5b701b0dbb6a15171)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:44.860268 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:44.866067 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:45.044078 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63067046807f42c5b701b0dbb6a15171'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:45.051471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63067046807f42c5b701b0dbb6a15171'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:45.052345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63067046807f42c5b701b0dbb6a15171'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:45.053269 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63067046807f42c5b701b0dbb6a15171'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:45.067937 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63067046807f42c5b701b0dbb6a15171'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:45.157906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1443/2839] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:02:44 2022] DELETE /identity/v3/users/63067046807f42c5b701b0dbb6a15171 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:45.162643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:45.163758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:45.163758 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:45.183214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:45.184471 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:45.255675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1397/2840] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:45.265578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:45.267054 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:45.330805 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57dd32-69ca-4f1f-b7c6-ae749d71b550 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:45.334380 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:45.335565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:45.335864 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:45.336137 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:45.345627 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:45.347069 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:45.356958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1444/2841] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:45.869488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:45.871003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:45.883610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b17b2-8486-41e5-9452-7982de3b92ad None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:45.888800 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:45.889443 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:45.889728 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:45.890013 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] PATH_INFO: `/v3/projects/b0dd5932001f4f618130198706a14d8b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:45.891277 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0dd5932001f4f618130198706a14d8b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:45.893140 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:46.045756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:46.049021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0dd5932001f4f618130198706a14d8b', 'request_id': 'req-858a6e52-a799-431e-883e-6ca2d1fd6477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.055318 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0dd5932001f4f618130198706a14d8b', 'request_id': 'req-858a6e52-a799-431e-883e-6ca2d1fd6477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.189359 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0dd5932001f4f618130198706a14d8b', 'request_id': 'req-858a6e52-a799-431e-883e-6ca2d1fd6477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.201304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] The token cache is being invalidate because project b0dd5932001f4f618130198706a14d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:46.201966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.202588 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.204647 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1398/2842] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:02:45 2022] DELETE /identity/v3/projects/b0dd5932001f4f618130198706a14d8b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:46.208675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:46.219807 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79676c30-365d-456d-ac36-d4cdafd4fe3c service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:46.283847 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:46.284503 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:46.284784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:46.285059 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] PATH_INFO: `/v3/users/cda50cefd1ec430cac2b23bf9310ef13` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:46.286240 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda50cefd1ec430cac2b23bf9310ef13)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:46.288567 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:46.393939 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:46.509253 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda50cefd1ec430cac2b23bf9310ef13'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.516592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda50cefd1ec430cac2b23bf9310ef13'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.517298 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda50cefd1ec430cac2b23bf9310ef13'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.517909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda50cefd1ec430cac2b23bf9310ef13'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.530992 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b1e9db7-709d-4558-bafb-5c8acddd7f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda50cefd1ec430cac2b23bf9310ef13'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:46.609468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1445/2843] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:02:46 2022] DELETE /identity/v3/users/cda50cefd1ec430cac2b23bf9310ef13 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:47.073794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.079990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:47.088771 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6e52-a799-431e-883e-6ca2d1fd6477 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.146565 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.149006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.149006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.149006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] PATH_INFO: `/v3/projects/29c1ff9e8a9e4443a33e9ba60d1d947f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.149427 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c1ff9e8a9e4443a33e9ba60d1d947f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.151174 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.256488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c1ff9e8a9e4443a33e9ba60d1d947f', 'request_id': 'req-0a991b64-36e3-4012-b17a-1d419ce74ec0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.263259 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c1ff9e8a9e4443a33e9ba60d1d947f', 'request_id': 'req-0a991b64-36e3-4012-b17a-1d419ce74ec0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.377747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c1ff9e8a9e4443a33e9ba60d1d947f', 'request_id': 'req-0a991b64-36e3-4012-b17a-1d419ce74ec0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.389246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] The token cache is being invalidate because project 29c1ff9e8a9e4443a33e9ba60d1d947f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:47.389534 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.390187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0a991b64-36e3-4012-b17a-1d419ce74ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.391852 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1399/2844] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:02:47 2022] DELETE /identity/v3/projects/29c1ff9e8a9e4443a33e9ba60d1d947f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:47.396754 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72094b7-65bf-44cc-b5d7-83028661383b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.397272 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72094b7-65bf-44cc-b5d7-83028661383b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.397365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72094b7-65bf-44cc-b5d7-83028661383b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.418762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e72094b7-65bf-44cc-b5d7-83028661383b None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:47.420024 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e72094b7-65bf-44cc-b5d7-83028661383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:47.481318 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1446/2845] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:47.488959 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194730a-14d5-4638-9523-9bd878c16a3a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.489251 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194730a-14d5-4638-9523-9bd878c16a3a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.489535 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194730a-14d5-4638-9523-9bd878c16a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.510702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e194730a-14d5-4638-9523-9bd878c16a3a None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:47.511983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e194730a-14d5-4638-9523-9bd878c16a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:47.520253 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1400/2846] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:47.524690 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72094b7-65bf-44cc-b5d7-83028661383b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.534977 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72094b7-65bf-44cc-b5d7-83028661383b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.540673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.541317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.541603 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.541883 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.542596 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:47.542906 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.544565 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.558248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1447/2847] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:47.562765 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194730a-14d5-4638-9523-9bd878c16a3a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.572928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194730a-14d5-4638-9523-9bd878c16a3a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.577847 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.578455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.578742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.579021 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.579752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.581679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.734585 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1401/2848] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:02:47 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:47.738798 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.749667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca0831-4a46-4717-8f94-d41b4a875fd3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.806254 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.807046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.807371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.807652 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.808340 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.810263 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.901673 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1448/2849] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:02:47 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:47.905741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.918056 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f12dd-98b3-462a-83b5-b6f9637be066 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.922818 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.926927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.926927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.926927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.926927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.926927 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.937679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1402/2850] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:47.941943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.952527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626674d-b879-4eba-8fc1-621399ee2ed0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.957317 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d83de-2766-4b4e-b17b-e0f5f867c315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.957928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d83de-2766-4b4e-b17b-e0f5f867c315 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.958211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d83de-2766-4b4e-b17b-e0f5f867c315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.958487 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d83de-2766-4b4e-b17b-e0f5f867c315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.959199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d83de-2766-4b4e-b17b-e0f5f867c315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.961314 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d83de-2766-4b4e-b17b-e0f5f867c315 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.972323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1449/2851] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:47.977194 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.987421 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4919a-af4f-445d-b4e3-1a977e4eb8ce admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.992068 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.992727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.993011 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.993287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] PATH_INFO: `/v3/projects/129a098922c54f4f8196f364c1538a8f/users/1daf92360738428db61049ad0dc09f53/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:48.007917 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129a098922c54f4f8196f364c1538a8f, user_id=1daf92360738428db61049ad0dc09f53, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:48.010591 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:48.073779 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:48.078893 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1403/2852] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:02:47 2022] PUT /identity/v3/projects/129a098922c54f4f8196f364c1538a8f/users/1daf92360738428db61049ad0dc09f53/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:48.084060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09916b6-e787-4ee5-b6ae-b157f32a098f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:48.084349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09916b6-e787-4ee5-b6ae-b157f32a098f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:48.084625 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09916b6-e787-4ee5-b6ae-b157f32a098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:48.105296 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e09916b6-e787-4ee5-b6ae-b157f32a098f None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:48.106526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e09916b6-e787-4ee5-b6ae-b157f32a098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:48.111801 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:48.167528 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1450/2853] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:48.177832 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:48.179670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:48.244142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f10012-7cb0-40cb-9299-23ddb884db48 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:48.244741 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:48.245372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:48.245661 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:48.245942 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:48.255589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:48.257044 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:48.266592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1404/2854] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:49.588307 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ff76-711f-4147-b0f0-56f85006d3d3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:49.589048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ff76-711f-4147-b0f0-56f85006d3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:49.589048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ff76-711f-4147-b0f0-56f85006d3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:49.620446 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5ab8ff76-711f-4147-b0f0-56f85006d3d3 None None] MFA Rules not processed for user `1daf92360738428db61049ad0dc09f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:49.621902 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5ab8ff76-711f-4147-b0f0-56f85006d3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:49.693049 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1451/2855] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:49.694247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:49.704033 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225ca2a-7db5-435f-8e81-52b990d21b1c service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:49.708541 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:49.709155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:49.709436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:49.709710 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:49.710407 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:49.712897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:49.822937 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1405/2856] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:02:49 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:49.827201 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ff76-711f-4147-b0f0-56f85006d3d3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:49.839124 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ff76-711f-4147-b0f0-56f85006d3d3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:49.899333 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:49.899961 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:49.900245 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:49.900521 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:49.901193 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:49.903176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.025354 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1452/2857] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:02:49 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:50.029468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.040908 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b108-97c0-4a92-a269-2b77995b58f9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.045502 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.046225 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.046505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.046781 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.047533 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.049287 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.060521 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1406/2858] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:50.068738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.079573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6e25c-7d1b-4fab-82b8-035d093c9174 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.084660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.085285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.085574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.086493 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] PATH_INFO: `/v3/projects/da84a848031a4b03ae1a86f60a6e9b0e/users/77ee34b1b96c4558adf46d614cd8a1a2/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.102341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=da84a848031a4b03ae1a86f60a6e9b0e, user_id=77ee34b1b96c4558adf46d614cd8a1a2, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.105070 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.184919 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1453/2859] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:02:50 2022] PUT /identity/v3/projects/da84a848031a4b03ae1a86f60a6e9b0e/users/77ee34b1b96c4558adf46d614cd8a1a2/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:50.189045 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.200346 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e076-7516-4ac6-8fb5-8b772e898f06 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.257027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92084610-3d54-4eb1-b97b-dcb07bc2bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.258310 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92084610-3d54-4eb1-b97b-dcb07bc2bf90 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.258596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92084610-3d54-4eb1-b97b-dcb07bc2bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.258872 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92084610-3d54-4eb1-b97b-dcb07bc2bf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.259596 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92084610-3d54-4eb1-b97b-dcb07bc2bf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.261192 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92084610-3d54-4eb1-b97b-dcb07bc2bf90 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.270807 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1407/2860] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:50.275109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.285478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca191-a403-4c4c-8b68-86e9ff247dbf admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.290359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.291030 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.291352 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.291630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] PATH_INFO: `/v3/projects/da84a848031a4b03ae1a86f60a6e9b0e/users/77ee34b1b96c4558adf46d614cd8a1a2/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.293607 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da84a848031a4b03ae1a86f60a6e9b0e, user_id=77ee34b1b96c4558adf46d614cd8a1a2, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.296282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.298678 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:50.308258 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1454/2861] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:02:50 2022] PUT /identity/v3/projects/da84a848031a4b03ae1a86f60a6e9b0e/users/77ee34b1b96c4558adf46d614cd8a1a2/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:51.649587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:51.649587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:51.649587 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:51.675405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] MFA Rules not processed for user `77ee34b1b96c4558adf46d614cd8a1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:51.676795 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:51.734006 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:51.753759 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1408/2862] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:51.765819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:51.767724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:51.828161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb528b3a-3913-4240-b290-afbb7306fd31 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:51.831699 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:51.832330 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:51.832616 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:51.832894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:51.890863 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:51.892341 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:51.901440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1455/2863] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:53.570905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.572374 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:53.584988 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc888e-3930-4ac5-a31a-b17174c8f254 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.588705 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.589347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.589629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.589909 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.600037 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.602134 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.612473 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1409/2864] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:02:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:01.897323 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=1,keystone:SELECT=50 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:03.607912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:09.351476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:09.355904 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:09.437227 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bb277-d360-45a7-9161-4c2c1c95856a service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:09.442917 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:09.443608 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:09.443901 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:09.444178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:09.446499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:09.458928 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:09.460572 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:09.471716 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1456/2865] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:10.182404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.251756 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561257ef-5c29-4dba-af92-24c4ade6aed4 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.256343 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.256986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.257274 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.257556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] PATH_INFO: `/v3/users/1daf92360738428db61049ad0dc09f53` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.258729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1daf92360738428db61049ad0dc09f53)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.260699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.432859 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1daf92360738428db61049ad0dc09f53'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.433452 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1daf92360738428db61049ad0dc09f53'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.440687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1daf92360738428db61049ad0dc09f53'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.507035 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1daf92360738428db61049ad0dc09f53'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.521497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1daf92360738428db61049ad0dc09f53'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.528643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1410/2866] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:03:10 2022] DELETE /identity/v3/users/1daf92360738428db61049ad0dc09f53 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:10.808498 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.819545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29743f-d345-4f8e-b4d4-3f284c8ccddb service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.876668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.877312 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.877602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.877880 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] PATH_INFO: `/v3/projects/129a098922c54f4f8196f364c1538a8f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.879032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129a098922c54f4f8196f364c1538a8f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.880976 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.980439 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129a098922c54f4f8196f364c1538a8f', 'request_id': 'req-a87f6988-0c2f-4293-864a-1fc596355066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.035092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129a098922c54f4f8196f364c1538a8f', 'request_id': 'req-a87f6988-0c2f-4293-864a-1fc596355066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.041411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129a098922c54f4f8196f364c1538a8f', 'request_id': 'req-a87f6988-0c2f-4293-864a-1fc596355066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.053001 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] The token cache is being invalidate because project 129a098922c54f4f8196f364c1538a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:11.053244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.053921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.055496 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1457/2867] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:03:10 2022] DELETE /identity/v3/projects/129a098922c54f4f8196f364c1538a8f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:11.059478 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.060977 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:11.074590 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefdbe1-1d19-452f-ac91-104881a53609 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.134263 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.134911 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.135228 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.135572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] PATH_INFO: `/v3/users/77ee34b1b96c4558adf46d614cd8a1a2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.136749 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ee34b1b96c4558adf46d614cd8a1a2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.138675 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.160794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:11.281413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ee34b1b96c4558adf46d614cd8a1a2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.282314 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ee34b1b96c4558adf46d614cd8a1a2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.289555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ee34b1b96c4558adf46d614cd8a1a2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.372687 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ee34b1b96c4558adf46d614cd8a1a2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.386112 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7aa283d-8566-4710-8822-36e210675a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ee34b1b96c4558adf46d614cd8a1a2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.392073 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1411/2868] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:03:11 2022] DELETE /identity/v3/users/77ee34b1b96c4558adf46d614cd8a1a2 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:11.660211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.661885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:11.678198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f6988-0c2f-4293-864a-1fc596355066 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.698837 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:11.740527 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.741183 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.741471 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.741759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] PATH_INFO: `/v3/projects/da84a848031a4b03ae1a86f60a6e9b0e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.743768 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da84a848031a4b03ae1a86f60a6e9b0e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.745724 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.870839 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da84a848031a4b03ae1a86f60a6e9b0e', 'request_id': 'req-26d8e6a6-e056-45b8-97b5-9660191a6873'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.938223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da84a848031a4b03ae1a86f60a6e9b0e', 'request_id': 'req-26d8e6a6-e056-45b8-97b5-9660191a6873'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.944118 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da84a848031a4b03ae1a86f60a6e9b0e', 'request_id': 'req-26d8e6a6-e056-45b8-97b5-9660191a6873'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.956953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] The token cache is being invalidate because project da84a848031a4b03ae1a86f60a6e9b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:11.957247 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.957895 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-26d8e6a6-e056-45b8-97b5-9660191a6873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:11.959547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1458/2869] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:03:11 2022] DELETE /identity/v3/projects/da84a848031a4b03ae1a86f60a6e9b0e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:11.964874 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92ca8e-6c25-4965-82be-b7b5e348e47e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.965169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92ca8e-6c25-4965-82be-b7b5e348e47e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.965528 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92ca8e-6c25-4965-82be-b7b5e348e47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.985706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9b92ca8e-6c25-4965-82be-b7b5e348e47e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:11.986932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9b92ca8e-6c25-4965-82be-b7b5e348e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:12.047432 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1412/2870] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:12.055345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4a13-3845-4729-89e0-7dbec7a12bca None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.055639 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4a13-3845-4729-89e0-7dbec7a12bca None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.055923 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4a13-3845-4729-89e0-7dbec7a12bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.075588 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-729b4a13-3845-4729-89e0-7dbec7a12bca None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:12.076877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-729b4a13-3845-4729-89e0-7dbec7a12bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:12.085015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1459/2871] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:12.090932 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92ca8e-6c25-4965-82be-b7b5e348e47e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.100976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92ca8e-6c25-4965-82be-b7b5e348e47e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.106284 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.106912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.107218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.107508 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.108214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:12.108527 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.110151 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:12.122457 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1413/2872] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:03:12.126819 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4a13-3845-4729-89e0-7dbec7a12bca None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.137071 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4a13-3845-4729-89e0-7dbec7a12bca None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.141762 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.142379 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.142663 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.142938 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.143647 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.145546 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:12.225753 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1460/2873] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:03:12 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:12.230193 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.241475 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd010b5f-72e1-42fd-a8e3-28eeb1ca3071 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.299690 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.300395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.300615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.300897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.301555 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.303511 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:12.395653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1414/2874] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:03:12 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:12.400045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.411102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bf3e6-ecdc-4f36-84f8-c6bfaf4a1415 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.416111 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.416757 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.417037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.417309 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.417997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.419615 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:12.430792 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1461/2875] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:03:12.434821 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.445088 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428f1ef-5394-409b-aa40-10ea5a1abeff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.450247 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97388364-acbf-4d5c-90ff-f2e1da5c73f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.450955 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97388364-acbf-4d5c-90ff-f2e1da5c73f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.451152 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97388364-acbf-4d5c-90ff-f2e1da5c73f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.451474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97388364-acbf-4d5c-90ff-f2e1da5c73f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.452177 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97388364-acbf-4d5c-90ff-f2e1da5c73f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.453787 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97388364-acbf-4d5c-90ff-f2e1da5c73f5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:12.465246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1415/2876] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:12.471211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.481364 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fa2fd-4ca5-420e-9b78-dc38c67c1e66 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.485891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.486548 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.486829 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.487103 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] PATH_INFO: `/v3/projects/03e11331b8d74c4b87439d499b8cebfa/users/3043305936794c19821fe59be4888920/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.501667 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e11331b8d74c4b87439d499b8cebfa, user_id=3043305936794c19821fe59be4888920, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.504344 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:12.573547 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1462/2877] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:03:12 2022] PUT /identity/v3/projects/03e11331b8d74c4b87439d499b8cebfa/users/3043305936794c19821fe59be4888920/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:12.585480 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bacd4c-57c2-45ac-904d-a3907aa03cbc None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.585775 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bacd4c-57c2-45ac-904d-a3907aa03cbc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.586059 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bacd4c-57c2-45ac-904d-a3907aa03cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.594946 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:12.615169 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-80bacd4c-57c2-45ac-904d-a3907aa03cbc None None] MFA Rules not processed for user `3043305936794c19821fe59be4888920`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:12.616455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-80bacd4c-57c2-45ac-904d-a3907aa03cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:12.685663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1416/2878] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:12.697465 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.699178 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:12.756526 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:12.763886 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922552e-a863-452e-913c-3793747b8db6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.767948 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b11c6e-7357-4551-bed1-8e0d0ccd020b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.768578 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b11c6e-7357-4551-bed1-8e0d0ccd020b service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.771458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b11c6e-7357-4551-bed1-8e0d0ccd020b service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.771458 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b11c6e-7357-4551-bed1-8e0d0ccd020b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.778468 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b11c6e-7357-4551-bed1-8e0d0ccd020b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.780061 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b11c6e-7357-4551-bed1-8e0d0ccd020b service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:12.789739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1463/2879] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:12.811357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bacd4c-57c2-45ac-904d-a3907aa03cbc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:12.873983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bacd4c-57c2-45ac-904d-a3907aa03cbc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:12.878571 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:12.879236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:12.879525 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:12.879804 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] PATH_INFO: `/v3/users/3043305936794c19821fe59be4888920` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:12.881142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3043305936794c19821fe59be4888920)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:12.883738 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:13.093339 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3043305936794c19821fe59be4888920'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.093929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3043305936794c19821fe59be4888920'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.100980 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3043305936794c19821fe59be4888920'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.167046 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3043305936794c19821fe59be4888920'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.179880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3043305936794c19821fe59be4888920'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.186466 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1417/2880] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:03:12 2022] DELETE /identity/v3/users/3043305936794c19821fe59be4888920 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:13.191251 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:13.191747 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:13.191836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:13.210338 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:13.211590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:13.270299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1464/2881] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:13.280130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:13.281549 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:13.296572 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:13.341019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a09b64-e866-419a-a976-e12647429829 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:13.344900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c584-92f7-4bd0-a027-600f3bd2e21d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:13.345472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c584-92f7-4bd0-a027-600f3bd2e21d service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:13.345757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c584-92f7-4bd0-a027-600f3bd2e21d service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:13.346036 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c584-92f7-4bd0-a027-600f3bd2e21d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:13.356164 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c584-92f7-4bd0-a027-600f3bd2e21d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:13.357689 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c584-92f7-4bd0-a027-600f3bd2e21d service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:13.368034 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1418/2882] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:13.773958 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:13.776936 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:13.788791 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc8834-dd5d-4f48-a5c4-c0813e5fdc7c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:13.794014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:13.794810 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:13.795102 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:13.795417 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] PATH_INFO: `/v3/projects/03e11331b8d74c4b87439d499b8cebfa` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:13.796630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e11331b8d74c4b87439d499b8cebfa)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:13.798566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:13.887692 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e11331b8d74c4b87439d499b8cebfa', 'request_id': 'req-187ed728-b05d-4ce5-8457-9607be6b0445'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.978402 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e11331b8d74c4b87439d499b8cebfa', 'request_id': 'req-187ed728-b05d-4ce5-8457-9607be6b0445'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.984914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e11331b8d74c4b87439d499b8cebfa', 'request_id': 'req-187ed728-b05d-4ce5-8457-9607be6b0445'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.996512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] The token cache is being invalidate because project 03e11331b8d74c4b87439d499b8cebfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:13.996812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.997495 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-187ed728-b05d-4ce5-8457-9607be6b0445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:13.999031 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1465/2883] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:03:13 2022] DELETE /identity/v3/projects/03e11331b8d74c4b87439d499b8cebfa => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:14.004060 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcb207-bcac-4511-8896-58afd7cdba6e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.004362 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcb207-bcac-4511-8896-58afd7cdba6e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.004653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcb207-bcac-4511-8896-58afd7cdba6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.026025 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-26fcb207-bcac-4511-8896-58afd7cdba6e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:14.027304 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-26fcb207-bcac-4511-8896-58afd7cdba6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:14.104175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1419/2884] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:14.112060 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe119-1298-4ecb-a9c5-64e30e1c3d73 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.112448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe119-1298-4ecb-a9c5-64e30e1c3d73 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.112836 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe119-1298-4ecb-a9c5-64e30e1c3d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.133862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-447fe119-1298-4ecb-a9c5-64e30e1c3d73 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:14.135740 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-447fe119-1298-4ecb-a9c5-64e30e1c3d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:14.143783 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1466/2885] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:14.148126 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcb207-bcac-4511-8896-58afd7cdba6e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:14.157968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcb207-bcac-4511-8896-58afd7cdba6e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:14.162760 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:14.163416 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.163703 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.163983 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.164694 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:14.165003 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:14.166654 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:14.178474 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1420/2886] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:03:14.183480 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe119-1298-4ecb-a9c5-64e30e1c3d73 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:14.192668 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe119-1298-4ecb-a9c5-64e30e1c3d73 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:14.197858 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:14.198461 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.198742 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.199015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.199706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:14.201621 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:14.315101 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1467/2887] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:03:14 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:14.320144 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:14.332264 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5ce4a-6a03-4b97-a86f-563234b3ccff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:14.395084 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eea497-8595-4958-8388-550904142930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:14.395752 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eea497-8595-4958-8388-550904142930 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.396019 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eea497-8595-4958-8388-550904142930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.396329 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eea497-8595-4958-8388-550904142930 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.399342 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eea497-8595-4958-8388-550904142930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:14.401629 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eea497-8595-4958-8388-550904142930 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:14.536206 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1421/2888] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:03:14 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:14.541260 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:14.552292 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf2c8-fede-429d-a670-67f01efa2f35 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:14.557838 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:14.563845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.563845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.563845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.563845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:14.563845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:14.572445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1468/2889] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:14.576656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eea497-8595-4958-8388-550904142930 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:14.587372 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eea497-8595-4958-8388-550904142930 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:14.592712 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d3685-be9c-43ee-85c6-f84023aa790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:14.593356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d3685-be9c-43ee-85c6-f84023aa790a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.593616 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d3685-be9c-43ee-85c6-f84023aa790a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.593897 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d3685-be9c-43ee-85c6-f84023aa790a admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.594593 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d3685-be9c-43ee-85c6-f84023aa790a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:14.596242 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d3685-be9c-43ee-85c6-f84023aa790a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:14.607120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1422/2890] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:14.611418 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:14.622212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2ee1c-5002-4330-93f2-c698b2a9dfa1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:14.626784 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:14.627478 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.627812 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.628048 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] PATH_INFO: `/v3/projects/89d23026f58a416a92daef9e220f3f1f/users/b5016785f3a44f6d9833ec8e1bd22e9f/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.643717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d23026f58a416a92daef9e220f3f1f, user_id=b5016785f3a44f6d9833ec8e1bd22e9f, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:14.646488 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:14.721472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1469/2891] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:03:14 2022] PUT /identity/v3/projects/89d23026f58a416a92daef9e220f3f1f/users/b5016785f3a44f6d9833ec8e1bd22e9f/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:14.733027 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6fae-2469-4256-83e8-8b8107fc43ef None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.733319 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6fae-2469-4256-83e8-8b8107fc43ef None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.733600 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6fae-2469-4256-83e8-8b8107fc43ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.771639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-62fc6fae-2469-4256-83e8-8b8107fc43ef None None] MFA Rules not processed for user `b5016785f3a44f6d9833ec8e1bd22e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:14.772883 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-62fc6fae-2469-4256-83e8-8b8107fc43ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:14.783136 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:14.837698 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1423/2892] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:14.842276 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:14.852426 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:14.904104 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622628-ad19-424d-8c2d-a536d75ed9dd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:14.908835 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:14.909451 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:14.909739 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:14.910025 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:14.910698 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:14.913139 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:15.034574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1470/2893] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:03:14 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:15.040115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc6fae-2469-4256-83e8-8b8107fc43ef None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:15.052067 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc6fae-2469-4256-83e8-8b8107fc43ef None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:15.108940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:15.109578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:15.109861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:15.110137 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:15.110814 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:15.112896 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:15.239254 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1424/2894] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:03:15 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:15.243482 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:15.254171 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe577-20a9-469b-974e-08b67174dbb7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:15.258893 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:15.259556 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:15.259967 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:15.260165 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:15.261428 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:15.263050 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:15.276099 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1471/2895] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:15.284299 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:15.294602 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8466b-8bb7-48d1-96d4-b3c4af20abd8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:15.299363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:15.300069 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:15.300356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:15.300633 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] PATH_INFO: `/v3/projects/ceac45b518b444f087756c86ba3a2fa0/users/7a6f867433904daf83c9aa3340d692fc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:15.315486 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceac45b518b444f087756c86ba3a2fa0, user_id=7a6f867433904daf83c9aa3340d692fc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:15.318218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:15.384385 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1425/2896] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:03:15 2022] PUT /identity/v3/projects/ceac45b518b444f087756c86ba3a2fa0/users/7a6f867433904daf83c9aa3340d692fc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:15.389719 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:15.400057 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7759475-b5f5-400d-b058-da7bcc142bc9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:15.458566 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c15072-4257-40e6-8ccb-43b2a8943a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:15.459225 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c15072-4257-40e6-8ccb-43b2a8943a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:15.459512 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c15072-4257-40e6-8ccb-43b2a8943a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:15.459793 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c15072-4257-40e6-8ccb-43b2a8943a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:15.460497 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c15072-4257-40e6-8ccb-43b2a8943a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:15.462109 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c15072-4257-40e6-8ccb-43b2a8943a6e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:15.473130 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1472/2897] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:15.477324 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:15.488170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87b51b-dba2-4dc9-bad0-84aaeaf581f7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:15.493028 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:15.493643 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:15.493934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:15.494215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] PATH_INFO: `/v3/projects/ceac45b518b444f087756c86ba3a2fa0/users/7a6f867433904daf83c9aa3340d692fc/roles/5d7ec7f52d40437c82607e4299a3bdd6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:15.496256 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceac45b518b444f087756c86ba3a2fa0, user_id=7a6f867433904daf83c9aa3340d692fc, role_id=5d7ec7f52d40437c82607e4299a3bdd6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:15.499224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:15.515727 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1426/2898] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:03:15 2022] PUT /identity/v3/projects/ceac45b518b444f087756c86ba3a2fa0/users/7a6f867433904daf83c9aa3340d692fc/roles/5d7ec7f52d40437c82607e4299a3bdd6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:15.527953 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c46e63-4d94-4ec7-b69f-ea1cc1a58524 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:15.528252 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c46e63-4d94-4ec7-b69f-ea1cc1a58524 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:15.528536 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c46e63-4d94-4ec7-b69f-ea1cc1a58524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:15.562590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-45c46e63-4d94-4ec7-b69f-ea1cc1a58524 None None] MFA Rules not processed for user `7a6f867433904daf83c9aa3340d692fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:15.563993 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-45c46e63-4d94-4ec7-b69f-ea1cc1a58524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:15.644499 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1473/2899] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:15.655444 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:15.657200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:15.720958 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ed793-7e92-4152-b680-2c23ae392145 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:15.724827 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b56a5-9b34-4129-946b-8716f5fefeaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:15.725451 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b56a5-9b34-4129-946b-8716f5fefeaf service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:15.725742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b56a5-9b34-4129-946b-8716f5fefeaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:15.726023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b56a5-9b34-4129-946b-8716f5fefeaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:15.743569 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:15.784384 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b56a5-9b34-4129-946b-8716f5fefeaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:15.785843 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b56a5-9b34-4129-946b-8716f5fefeaf service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:15.795128 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1427/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:18.902018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c46e63-4d94-4ec7-b69f-ea1cc1a58524 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.907705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:18.971764 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c46e63-4d94-4ec7-b69f-ea1cc1a58524 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.976389 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.980460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.980460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.980460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] PATH_INFO: `/v3/users/b5016785f3a44f6d9833ec8e1bd22e9f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.980460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5016785f3a44f6d9833ec8e1bd22e9f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.981356 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:19.201890 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5016785f3a44f6d9833ec8e1bd22e9f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.208820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5016785f3a44f6d9833ec8e1bd22e9f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.209492 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5016785f3a44f6d9833ec8e1bd22e9f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.210231 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5016785f3a44f6d9833ec8e1bd22e9f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.223143 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5016785f3a44f6d9833ec8e1bd22e9f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.298459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1474/2901] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:03:18 2022] DELETE /identity/v3/users/b5016785f3a44f6d9833ec8e1bd22e9f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:19.303214 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:19.303709 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:19.303972 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:19.325456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:19.326677 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:19.390311 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1428/2902] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:19.400032 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:19.401476 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:19.461780 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6591f05-4074-4835-87c9-2d54ef6612c1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:19.465626 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:19.466303 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:19.467045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:19.467045 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:19.477709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:19.479223 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:19.489293 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1475/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:19.967012 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:19.968567 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:19.982185 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3698f-a48b-48b8-935a-257252892e7e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:19.987207 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:19.988788 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:19.989082 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:19.989365 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] PATH_INFO: `/v3/projects/89d23026f58a416a92daef9e220f3f1f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:19.990556 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d23026f58a416a92daef9e220f3f1f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:19.992526 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:20.122493 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d23026f58a416a92daef9e220f3f1f', 'request_id': 'req-21197aa0-22ce-45fa-946a-e64f9747ded7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.128833 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d23026f58a416a92daef9e220f3f1f', 'request_id': 'req-21197aa0-22ce-45fa-946a-e64f9747ded7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.235357 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d23026f58a416a92daef9e220f3f1f', 'request_id': 'req-21197aa0-22ce-45fa-946a-e64f9747ded7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.248640 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] The token cache is being invalidate because project 89d23026f58a416a92daef9e220f3f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:20.248929 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.249595 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.251236 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1429/2904] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:03:19 2022] DELETE /identity/v3/projects/89d23026f58a416a92daef9e220f3f1f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:20.255400 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:20.266299 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54f3ab-517b-438a-8cef-99e9b1040bb0 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:20.291717 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:20.325017 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:20.325710 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:20.325996 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:20.326514 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] PATH_INFO: `/v3/users/7a6f867433904daf83c9aa3340d692fc` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:20.327730 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6f867433904daf83c9aa3340d692fc)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:20.329706 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:20.488409 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6f867433904daf83c9aa3340d692fc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.499261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6f867433904daf83c9aa3340d692fc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.500004 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6f867433904daf83c9aa3340d692fc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.500620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6f867433904daf83c9aa3340d692fc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.513789 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0a6d14b2-4578-4184-bd42-bdcb81ad41fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6f867433904daf83c9aa3340d692fc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.578133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1476/2905] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:03:20 2022] DELETE /identity/v3/users/7a6f867433904daf83c9aa3340d692fc => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:20.983213 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:20.984842 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:20.998477 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21197aa0-22ce-45fa-946a-e64f9747ded7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.040876 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:21.057420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.058072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.058363 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.058651 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] PATH_INFO: `/v3/projects/ceac45b518b444f087756c86ba3a2fa0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.059851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceac45b518b444f087756c86ba3a2fa0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.061780 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.192851 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceac45b518b444f087756c86ba3a2fa0', 'request_id': 'req-784445ec-d417-4385-84d5-bcd7c76711fa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.199238 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceac45b518b444f087756c86ba3a2fa0', 'request_id': 'req-784445ec-d417-4385-84d5-bcd7c76711fa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.240607 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceac45b518b444f087756c86ba3a2fa0', 'request_id': 'req-784445ec-d417-4385-84d5-bcd7c76711fa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.253763 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] The token cache is being invalidate because project ceac45b518b444f087756c86ba3a2fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:21.254055 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.254742 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-784445ec-d417-4385-84d5-bcd7c76711fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.256376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1430/2906] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:03:20 2022] DELETE /identity/v3/projects/ceac45b518b444f087756c86ba3a2fa0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:21.261877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ebc4f-232e-4ef5-9808-21ea9174d5f4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.262359 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ebc4f-232e-4ef5-9808-21ea9174d5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.262459 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ebc4f-232e-4ef5-9808-21ea9174d5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.284665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-110ebc4f-232e-4ef5-9808-21ea9174d5f4 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:21.285885 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-110ebc4f-232e-4ef5-9808-21ea9174d5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:21.347119 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1477/2907] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:21.354940 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9e5f6-6bfe-4f3e-8dfd-3d5129799f1d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.355255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9e5f6-6bfe-4f3e-8dfd-3d5129799f1d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.355537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9e5f6-6bfe-4f3e-8dfd-3d5129799f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.375130 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-58c9e5f6-6bfe-4f3e-8dfd-3d5129799f1d None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:21.376434 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-58c9e5f6-6bfe-4f3e-8dfd-3d5129799f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:21.384387 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1431/2908] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:21.389122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ebc4f-232e-4ef5-9808-21ea9174d5f4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:21.399029 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ebc4f-232e-4ef5-9808-21ea9174d5f4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.404188 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.404797 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.405080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.405351 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.406039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:21.406345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.408046 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.420179 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1478/2909] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:03:21.424574 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9e5f6-6bfe-4f3e-8dfd-3d5129799f1d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:21.433982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9e5f6-6bfe-4f3e-8dfd-3d5129799f1d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.439022 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.439679 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.439966 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.440249 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.440922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.442845 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.537520 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1432/2910] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:03:21 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:21.542570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:21.553088 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af329b75-47e3-422e-93b7-a09a12bf3c9a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.585803 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:21.615817 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.616506 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.616933 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.617472 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.618371 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.621365 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.704769 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1479/2911] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:03:21 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:21.708890 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:21.719143 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ab816-e823-431b-bab4-e63712e86491 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.723998 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.724639 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.724926 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.725265 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.725968 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.727706 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.738070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1433/2912] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:03:21.742147 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:21.753121 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361156d-ab80-408c-83b3-198a27b8e46d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.757914 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e060e89-690a-43b7-899f-4bce75f12bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.758545 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e060e89-690a-43b7-899f-4bce75f12bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.758834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e060e89-690a-43b7-899f-4bce75f12bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.759112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e060e89-690a-43b7-899f-4bce75f12bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.759845 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e060e89-690a-43b7-899f-4bce75f12bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.762018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e060e89-690a-43b7-899f-4bce75f12bb1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.772429 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1480/2913] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:21.776773 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:21.787080 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574b9d-5a28-4939-b84a-fa0d046bbcec admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.792215 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.792887 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.793170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.793450 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] PATH_INFO: `/v3/projects/6eeb5f1a95554adfbd102b83eabf0063/users/41b50dacc23d4727908617ba48de91bd/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.808729 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eeb5f1a95554adfbd102b83eabf0063, user_id=41b50dacc23d4727908617ba48de91bd, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.811423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.887227 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1434/2914] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:03:21 2022] PUT /identity/v3/projects/6eeb5f1a95554adfbd102b83eabf0063/users/41b50dacc23d4727908617ba48de91bd/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:21.897570 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4892372-8080-4521-868e-8a4a12a019a3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.897860 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4892372-8080-4521-868e-8a4a12a019a3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.898142 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4892372-8080-4521-868e-8a4a12a019a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.926982 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d4892372-8080-4521-868e-8a4a12a019a3 None None] MFA Rules not processed for user `41b50dacc23d4727908617ba48de91bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:21.928285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d4892372-8080-4521-868e-8a4a12a019a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:21.992042 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1481/2915] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:22.003957 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:22.005688 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:22.063120 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87f731-4239-4174-8043-c0fc1e548377 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:22.066747 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7285e-0cb9-419e-83ba-2f5753e97f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:22.067405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7285e-0cb9-419e-83ba-2f5753e97f5d service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.067702 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7285e-0cb9-419e-83ba-2f5753e97f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.067982 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7285e-0cb9-419e-83ba-2f5753e97f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.076334 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7285e-0cb9-419e-83ba-2f5753e97f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:22.080986 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7285e-0cb9-419e-83ba-2f5753e97f5d service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:22.086922 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1435/2916] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:31.988972 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:32.082944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:46.164590 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4892372-8080-4521-868e-8a4a12a019a3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:46.225620 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4892372-8080-4521-868e-8a4a12a019a3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:46.230106 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:46.230738 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:46.231022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:46.231325 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] PATH_INFO: `/v3/users/41b50dacc23d4727908617ba48de91bd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:46.233084 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b50dacc23d4727908617ba48de91bd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:46.235036 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:46.419115 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b50dacc23d4727908617ba48de91bd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:46.426733 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b50dacc23d4727908617ba48de91bd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:46.427441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b50dacc23d4727908617ba48de91bd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:46.428074 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b50dacc23d4727908617ba48de91bd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:46.439828 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b50dacc23d4727908617ba48de91bd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:46.527448 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1482/2917] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:03:46 2022] DELETE /identity/v3/users/41b50dacc23d4727908617ba48de91bd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:46.532516 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:46.533007 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:46.533204 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:46.553142 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:46.554356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:46.612440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1436/2918] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:46.622138 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:46.623613 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:46.681432 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c46084-c404-4000-bb79-45f883124e51 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:46.685018 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b7825-89af-4b69-ab63-a9f11b491e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:46.685654 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7825-89af-4b69-ab63-a9f11b491e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:46.685940 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7825-89af-4b69-ab63-a9f11b491e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:46.686217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7825-89af-4b69-ab63-a9f11b491e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:46.695629 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b7825-89af-4b69-ab63-a9f11b491e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:46.697177 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b7825-89af-4b69-ab63-a9f11b491e96 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:46.707694 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1483/2919] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:47.157990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:47.160836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:47.173289 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eee76a-e233-4aa8-93db-7346feb448f3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:47.178456 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:47.179115 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.179488 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.179770 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] PATH_INFO: `/v3/projects/6eeb5f1a95554adfbd102b83eabf0063` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.180999 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eeb5f1a95554adfbd102b83eabf0063)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:47.182923 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:47.312029 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eeb5f1a95554adfbd102b83eabf0063', 'request_id': 'req-2db7b694-1636-4dc2-88f4-9eeefa2a7611'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:47.318276 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eeb5f1a95554adfbd102b83eabf0063', 'request_id': 'req-2db7b694-1636-4dc2-88f4-9eeefa2a7611'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:47.385744 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eeb5f1a95554adfbd102b83eabf0063', 'request_id': 'req-2db7b694-1636-4dc2-88f4-9eeefa2a7611'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:47.400072 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] The token cache is being invalidate because project 6eeb5f1a95554adfbd102b83eabf0063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:47.404902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:47.404902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2db7b694-1636-4dc2-88f4-9eeefa2a7611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:47.404902 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1437/2920] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:03:47 2022] DELETE /identity/v3/projects/6eeb5f1a95554adfbd102b83eabf0063 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:47.408831 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f2cc-ac13-4f96-870c-beeb91e03945 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.409445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f2cc-ac13-4f96-870c-beeb91e03945 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.409602 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f2cc-ac13-4f96-870c-beeb91e03945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.432215 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f255f2cc-ac13-4f96-870c-beeb91e03945 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:47.433437 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f255f2cc-ac13-4f96-870c-beeb91e03945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:47.450592 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:47.493189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1484/2921] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:47.500891 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a26e5-3028-4735-817b-c858ed301942 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.501187 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a26e5-3028-4735-817b-c858ed301942 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.501468 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a26e5-3028-4735-817b-c858ed301942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.522599 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-947a26e5-3028-4735-817b-c858ed301942 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:47.524611 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-947a26e5-3028-4735-817b-c858ed301942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:47.532472 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1438/2922] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:47.536894 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255f2cc-ac13-4f96-870c-beeb91e03945 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:47.547776 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255f2cc-ac13-4f96-870c-beeb91e03945 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:47.552377 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:47.552997 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.553282 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.553563 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.554264 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:47.554574 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:47.556211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:47.567750 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1485/2923] 10.222.0.49 () {58 vars in 1153 bytes} [Tue Aug 2 11:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:03:47.572200 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a26e5-3028-4735-817b-c858ed301942 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:47.581670 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a26e5-3028-4735-817b-c858ed301942 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:47.587246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:47.587246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.587246 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.587578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.588275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:47.590173 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:47.666127 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1439/2924] 10.222.0.49 () {60 vars in 1153 bytes} [Tue Aug 2 11:03:47 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:47.671249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:47.682184 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157500-97a1-4f0e-b55e-b29577bd5572 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:47.739372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:47.740022 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.740311 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.741332 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.742015 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:47.744019 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:47.845198 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1486/2925] 10.222.0.49 () {60 vars in 1144 bytes} [Tue Aug 2 11:03:47 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:47.849332 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:47.859987 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ffb53-6982-4da3-a751-5af4989d872a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:47.864737 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:47.865356 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.865653 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.865928 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.866663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:47.868293 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:47.878764 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1440/2926] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:47.882763 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:47.893244 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1e87-4902-44a0-8a9a-83b5b865fd4e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:47.898660 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865d36b-dc4c-4853-882d-88b60287461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:47.899306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d36b-dc4c-4853-882d-88b60287461b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.899594 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d36b-dc4c-4853-882d-88b60287461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.899877 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d36b-dc4c-4853-882d-88b60287461b admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.900573 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865d36b-dc4c-4853-882d-88b60287461b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:47.902162 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865d36b-dc4c-4853-882d-88b60287461b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:47.913285 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1487/2927] 10.222.0.49 () {58 vars in 1122 bytes} [Tue Aug 2 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:47.917483 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:47.927699 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bc0f0-d396-400b-b286-4ab55471b407 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:47.932436 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:47.933101 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:47.933382 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:47.933656 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] PATH_INFO: `/v3/projects/5c7f14f32755499680d0678f349ad2aa/users/fcf55ad0572e4f7492ab7d73fa0cdce7/roles/3c92f2e9a9cc4210be2f0e54650bab15` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:47.950697 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7f14f32755499680d0678f349ad2aa, user_id=fcf55ad0572e4f7492ab7d73fa0cdce7, role_id=3c92f2e9a9cc4210be2f0e54650bab15)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:47.954032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:48.042548 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1441/2928] 10.222.0.49 () {60 vars in 1483 bytes} [Tue Aug 2 11:03:47 2022] PUT /identity/v3/projects/5c7f14f32755499680d0678f349ad2aa/users/fcf55ad0572e4f7492ab7d73fa0cdce7/roles/3c92f2e9a9cc4210be2f0e54650bab15 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:48.047642 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4b7c4-39f3-44b9-b04f-dcc66078d307 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:48.047932 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4b7c4-39f3-44b9-b04f-dcc66078d307 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:48.048212 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4b7c4-39f3-44b9-b04f-dcc66078d307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:48.066921 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d2a4b7c4-39f3-44b9-b04f-dcc66078d307 None None] MFA Rules not processed for user `f7156356453b494e8893e4adc4a9ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:48.068161 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d2a4b7c4-39f3-44b9-b04f-dcc66078d307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:48.127362 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1488/2929] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:48.137575 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:48.138632 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:48.161536 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:48.196834 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb1b40-d05f-4a98-ace2-e3a082354f91 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:48.200490 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:48.202420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:48.202420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:48.202420 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:48.211221 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:48.212684 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:48.222307 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1442/2930] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:49.508133 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe334d4-a849-415c-bf6e-f8391bbc7ce8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:49.511112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe334d4-a849-415c-bf6e-f8391bbc7ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:49.511112 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe334d4-a849-415c-bf6e-f8391bbc7ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:49.540978 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dbe334d4-a849-415c-bf6e-f8391bbc7ce8 None None] MFA Rules not processed for user `fcf55ad0572e4f7492ab7d73fa0cdce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:49.542217 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dbe334d4-a849-415c-bf6e-f8391bbc7ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:49.559441 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:49.611041 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1489/2931] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:49.621995 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:49.623497 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:49.681501 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fadf84-66c6-4568-92c6-29e9b74b5731 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:49.685224 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbb92bcc5054287a96fbfac03d2975a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:49.686336 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:49.686625 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:49.686912 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:49.695413 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:49.696880 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:49.706711 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1443/2932] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:51.475289 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783489eb-b672-4226-9ce3-7d097115f859 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:51.475806 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783489eb-b672-4226-9ce3-7d097115f859 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:51.475995 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783489eb-b672-4226-9ce3-7d097115f859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:51.495279 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-783489eb-b672-4226-9ce3-7d097115f859 None None] MFA Rules not processed for user `fcf55ad0572e4f7492ab7d73fa0cdce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:51.496518 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-783489eb-b672-4226-9ce3-7d097115f859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:51.504224 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1490/2933] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:03:51.548934 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:51.551155 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:51.618057 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae375c7e-5e0a-4e66-8f58-9e1e6ded667c service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:51.621748 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '700720d6f66e4de2a65b808afce70e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:51.622376 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:51.622660 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:51.622944 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:51.625505 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:51.706199 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:51.707624 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:51.713976 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:51.730014 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1444/2934] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:51.941092 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783489eb-b672-4226-9ce3-7d097115f859 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:51.951363 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783489eb-b672-4226-9ce3-7d097115f859 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:51.955460 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:51.956080 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:51.956413 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:51.956634 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:51.958540 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:51.969804 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:51.971248 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:51.980110 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1491/2935] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:52.694107 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:52.695603 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:52.700712 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:52.702249 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:52.713610 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e2786-5a91-417d-8464-b8269a779e27 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:52.718063 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:52.718682 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:52.718963 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:52.719274 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:52.721122 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:52.735820 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:52.737261 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:52.748688 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1492/2936] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:52.773598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9f852-febc-46a4-bb3d-60135d48c4ec service glance] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:52.778905 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:52.780023 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:52.780309 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:52.780592 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:52.782848 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:52.801255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:52.807032 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:52.818537 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1445/2937] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:56.027827 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:56.037176 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a69662-48a5-4547-9ce4-f4f77dbf78bd service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:56.041257 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:56.041870 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:56.042156 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:56.042434 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:56.044199 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:56.056259 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:56.057760 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:56.066433 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1493/2938] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:56.781885 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:56.787109 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:56.799676 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326d4e7-decd-4c8a-af5c-5622c4e3422b service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:56.805135 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acee02a-f43b-41c0-83bc-fdf55174ace0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:56.805794 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee02a-f43b-41c0-83bc-fdf55174ace0 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:56.806081 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee02a-f43b-41c0-83bc-fdf55174ace0 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:56.806361 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee02a-f43b-41c0-83bc-fdf55174ace0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:56.808405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1acee02a-f43b-41c0-83bc-fdf55174ace0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:56.828990 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acee02a-f43b-41c0-83bc-fdf55174ace0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:56.830610 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acee02a-f43b-41c0-83bc-fdf55174ace0 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:56.841861 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1446/2939] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:04:01.872630 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.886998 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf506489-26e9-4f27-828a-73145bbdf51c service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.895576 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9072775865604c73ae8361a047945464', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.896205 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.896677 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.896874 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.898734 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:01.911808 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.913445 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.924915 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1494/2940] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:04:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:06.837262 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=42 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:09.079355 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fce3-6c88-44b5-a499-e7fcc62f7cc6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:09.083721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fce3-6c88-44b5-a499-e7fcc62f7cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:09.083721 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fce3-6c88-44b5-a499-e7fcc62f7cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:09.099330 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-47e0fce3-6c88-44b5-a499-e7fcc62f7cc6 None None] MFA Rules not processed for user `fcf55ad0572e4f7492ab7d73fa0cdce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:09.100530 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-47e0fce3-6c88-44b5-a499-e7fcc62f7cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:09.107373 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1447/2941] 10.222.0.49 () {58 vars in 958 bytes} [Tue Aug 2 11:04:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:09.340238 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:09.349651 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3c17-1972-4028-a1ae-88fc5e5c7dbf service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:09.353295 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cabb184e704aecbb0a3bfb1cd0a584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:09.353903 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:09.354189 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:09.354469 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:09.356321 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:09.366014 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:09.367475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:09.375899 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1495/2942] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:04:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:19.093967 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:19.372221 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=52 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:37.433601 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb747d15-5adc-4dfd-b1a1-88b9097e16b5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:37.434070 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb747d15-5adc-4dfd-b1a1-88b9097e16b5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:37.434175 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb747d15-5adc-4dfd-b1a1-88b9097e16b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:37.497347 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cb747d15-5adc-4dfd-b1a1-88b9097e16b5 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:37.498598 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cb747d15-5adc-4dfd-b1a1-88b9097e16b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:37.567001 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1448/2943] 10.222.0.49 () {58 vars in 1009 bytes} [Tue Aug 2 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:37.712349 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:37.713908 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:37.803411 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1529e29-b6db-41bd-95ac-7927095e8c9b service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:37.808675 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875b396-b79f-4a1b-862a-e9823fa8c8df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:37.809306 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875b396-b79f-4a1b-862a-e9823fa8c8df service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:37.809605 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875b396-b79f-4a1b-862a-e9823fa8c8df service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:37.809873 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875b396-b79f-4a1b-862a-e9823fa8c8df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:37.818848 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875b396-b79f-4a1b-862a-e9823fa8c8df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:37.821435 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875b396-b79f-4a1b-862a-e9823fa8c8df service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:37.829033 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1496/2944] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 11:04:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:04:47.563615 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=44 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:47.825636 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=41 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:52.929739 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb747d15-5adc-4dfd-b1a1-88b9097e16b5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:52.943111 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb747d15-5adc-4dfd-b1a1-88b9097e16b5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:52.948255 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:52.948873 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:52.949160 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:52.949440 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:52.951218 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:52.963579 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:52.965163 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:52.975172 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1449/2945] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:04:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:54.794463 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8382cbb-5156-4502-94c1-434909955309 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:54.797345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8382cbb-5156-4502-94c1-434909955309 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:54.797345 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8382cbb-5156-4502-94c1-434909955309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:54.815909 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f8382cbb-5156-4502-94c1-434909955309 None None] MFA Rules not processed for user `a75c1093580244409a9074cc0d277af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:54.817676 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f8382cbb-5156-4502-94c1-434909955309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:54.825039 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1497/2946] 10.222.0.49 () {58 vars in 967 bytes} [Tue Aug 2 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:54.837961 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:54.850065 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930828d7-252e-41d1-9659-ab8dd2f22087 service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:54.855335 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f7db7eea94472c82cda44c569d9550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:54.855900 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:54.856181 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:54.856455 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:54.865404 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:54.868757 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:54.877275 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1450/2947] 10.222.0.49 () {60 vars in 1384 bytes} [Tue Aug 2 11:04:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:04:58.010462 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8382cbb-5156-4502-94c1-434909955309 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:58.027467 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8382cbb-5156-4502-94c1-434909955309 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:58.027727 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:58.028372 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:58.028851 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:58.029023 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] PATH_INFO: `/v3/users/fcf55ad0572e4f7492ab7d73fa0cdce7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:58.030145 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf55ad0572e4f7492ab7d73fa0cdce7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:58.032021 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:58.255062 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf55ad0572e4f7492ab7d73fa0cdce7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.262211 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf55ad0572e4f7492ab7d73fa0cdce7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.262891 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf55ad0572e4f7492ab7d73fa0cdce7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.263520 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf55ad0572e4f7492ab7d73fa0cdce7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.275475 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf55ad0572e4f7492ab7d73fa0cdce7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.352107 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1498/2948] 10.222.0.49 () {58 vars in 1224 bytes} [Tue Aug 2 11:04:58 2022] DELETE /identity/v3/users/fcf55ad0572e4f7492ab7d73fa0cdce7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:58.602627 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:58.613423 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2b9cf-6a7f-4b32-867d-92649bdbd6c5 service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:58.672184 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7156356453b494e8893e4adc4a9ad81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '737b03aeb0594b48a03adad9f790a347', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:58.672836 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:58.673124 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:58.673405 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] PATH_INFO: `/v3/projects/5c7f14f32755499680d0678f349ad2aa` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:58.674578 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7f14f32755499680d0678f349ad2aa)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:58.677170 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:58.840395 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7f14f32755499680d0678f349ad2aa', 'request_id': 'req-706ae302-9438-44f7-8950-613587825f62'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.846663 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7f14f32755499680d0678f349ad2aa', 'request_id': 'req-706ae302-9438-44f7-8950-613587825f62'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.942589 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7f14f32755499680d0678f349ad2aa', 'request_id': 'req-706ae302-9438-44f7-8950-613587825f62'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.954411 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] The token cache is being invalidate because project 5c7f14f32755499680d0678f349ad2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:58.954720 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.955412 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-706ae302-9438-44f7-8950-613587825f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.957017 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1451/2949] 10.222.0.49 () {58 vars in 1233 bytes} [Tue Aug 2 11:04:58 2022] DELETE /identity/v3/projects/5c7f14f32755499680d0678f349ad2aa => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:05:08.348100 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:08.951294 dv-cinder-847536-4-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:09:58.875705 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:09:58.877232 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:09:59.003037 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661002ce-a522-41db-bfde-6e6a22f9e918 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:09:59.064709 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2056b3-1f69-438e-bafb-dbb21ee20138 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68c847a5a60461a840679bbefc8ab6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa12cb147a104399bade5d920da1304b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:09:59.065378 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2056b3-1f69-438e-bafb-dbb21ee20138 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:09:59.065665 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2056b3-1f69-438e-bafb-dbb21ee20138 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:09:59.065943 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2056b3-1f69-438e-bafb-dbb21ee20138 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:09:59.067759 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0c2056b3-1f69-438e-bafb-dbb21ee20138 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:09:59.079834 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2056b3-1f69-438e-bafb-dbb21ee20138 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:09:59.081440 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2056b3-1f69-438e-bafb-dbb21ee20138 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:09:59.092265 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1499/2950] 10.222.0.49 () {60 vars in 1365 bytes} [Tue Aug 2 11:09:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:10:09.087862 dv-cinder-847536-4-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=80 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}